Search under uncertainty: Cognitive biases and heuristics: a tutorial on testing, mitigating and accounting for cognitive biases in search experiments

J Liu, L Azzopardi - Proceedings of the 47th International ACM SIGIR …, 2024 - dl.acm.org
Understanding how people interact with search interfaces is core to the field of Interactive
Information Retrieval (IIR). While various models have been proposed (eg, Belkin's ASK …

Search under Uncertainty: Cognitive Biases and Heuristics-Tutorial on Modeling Search Interaction using Behavioral Economics

L Azzopardi, J Liu - Proceedings of the 2024 Conference on Human …, 2024 - dl.acm.org
Modeling how people interact with search interfaces is core to the field of Interactive
Information Retrieval. While various models have been proposed ranging from conceptual …

Ai can be cognitively biased: An exploratory study on threshold priming in llm-based batch relevance assessment

N Chen, J Liu, X Dong, Q Liu, T Sakai… - Proceedings of the 2024 …, 2024 - dl.acm.org
Cognitive biases are systematic deviations in thinking that lead to irrational judgments and
problematic decision-making, extensively studied across various fields. Recently, large …

Cognitively Biased Users Interacting with Algorithmically Biased Results in Whole-Session Search on Debated Topics

B Wang, J Liu - Proceedings of the 2024 ACM SIGIR International …, 2024 - dl.acm.org
When interacting with information retrieval (IR) systems, users, affected by confirmation
biases, tend to select search results that confirm their existing beliefs on socially significant …

What matters in a measure? A perspective from large-scale search evaluation

P Thomas, G Kazai, N Craswell… - Proceedings of the 47th …, 2024 - dl.acm.org
Information retrieval (IR) has a large literature on evaluation, dating back decades and
forming a central part of the research culture. The largest proportion of this literature …

SWAN: A Generic Framework for Auditing Textual Conversational Systems

T Sakai - arxiv preprint arxiv:2305.08290, 2023 - arxiv.org
We present a simple and generic framework for auditing a given textual conversational
system, given some samples of its conversation sessions as its input. The framework …

Decoy Effect in Search Interaction: A Pilot Study

N Chen, J Liu, T Sakai, XM Wu - arxiv preprint arxiv:2311.02362, 2023 - arxiv.org
In recent years, the influence of cognitive effects and biases on users' thinking, behaving,
and decision-making has garnered increasing attention in the field of interactive information …

Understanding users' dynamic perceptions of search gain and cost in sessions: An expectation confirmation model

B Wang, J Liu - Journal of the Association for Information …, 2024 - Wiley Online Library
Understanding the roles of search gain and cost in users' search decision‐making is a key
topic in interactive information retrieval (IIR). While previous research has developed user …

Evaluating Conversational Recommender Systems with Large Language Models: A User-Centric Evaluation Framework

N Chen, Q Dai, X Dong, XM Wu, Z Dong - arxiv preprint arxiv:2501.09493, 2025 - arxiv.org
Conversational recommender systems (CRS) involve both recommendation and dialogue
tasks, which makes their evaluation a unique challenge. Although past research has …

Evaluating Cognitive Biases in Conversational and Generative IIR: A Tutorial

L Azzopardi, J Liu - Proceedings of the 2024 Annual International ACM …, 2024 - dl.acm.org
Understanding how and why people interact with information through search interfaces is
central to Interactive Information Retrieval (IIR). Research has demonstrated that cognitive …