With the emergence of various information access systems exhibiting increasing complexity,
there is a critical need for sound and scalable means of automatic evaluation. To address …

This paper outlines a conceptual framework for understanding recent developments in
information retrieval and natural language processing that attempts to integrate dense and …

Simulation is used as a low-cost and repeatable means of experimentation. As Information
Retrieval (IR) researchers, we are no strangers to the idea of using simulation within our …

Complex search tasks—such as those from the Search as Learning (SAL) domain—often
result in users develo** an information need composed of several aspects. However …

Simulated user retrieval system interactions enable studies with controlled user behavior. To
this end, the SimIIR framework offers static, rule-based methods. We present an extended …

Abstract System-oriented IR evaluations are limited to rather abstract understandings of real
user behavior. As a solution, simulating user interactions provides a cost-efficient way to …

While the Cranfield evaluation methodology based on test collections has been very useful
for evaluating simple IR systems that return a ranked list of documents, it has significant …

Most of the current models that are used to simulate users in Interactive Information Retrieval
(IIR) lack realism and agency. Such models generally make decisions in a stochastic …

In this perspective paper we study the effect of non independent and identically distributed
(non-IID) data on federated online learning to rank (FOLTR) and chart directions for future …

Considering the multimodal signals of search items is beneficial for retrieval effectiveness.
Especially in web table retrieval (WTR) experiments, accounting for multimodal properties of …