With the emergence of various information access systems exhibiting increasing complexity,
there is a critical need for sound and scalable means of automatic evaluation. To address …

This paper explores the utility of a Large Language Model (LLM) to automatically generate
queries and query variants from a description of an information need. Given a set of …

Due to the advantages in the cost-efficiency and reproducibility, user simulation has become
a promising solution to the user-centric evaluation of information retrieval systems …

Information seeking conversations between users and Conversational Search Agents
(CSAs) consist of multiple turns of interaction. While users initiate a search session, ideally a …

Searching is inherently an interactive process usually requiring numerous iterations of
querying and assessing in order to find the desired amount of relevant information …

There has been increased interest in the use of simulated queries for evaluation and
estimation purposes in Information Retrieval. However, there are still many unaddressed …

Information retrieval (IR) evaluation scores are generally designed to measure the
effectiveness with which relevant documents are identified and retrieved. Many scores have …

Interactive query formulation | Annual Review of Information Science and Technology skip to
main content ACM Digital Library home ACM Association for Computing Machinery corporate …

Simulated user retrieval system interactions enable studies with controlled user behavior. To
this end, the SimIIR framework offers static, rule-based methods. We present an extended …

While the Cranfield evaluation methodology based on test collections has been very useful
for evaluating simple IR systems that return a ranked list of documents, it has significant …