Recommendation, information retrieval, and other information access systems pose unique
challenges for investigating and applying the fairness and non-discrimination concepts that …

Recent work has advocated for training AI models on ever-larger datasets, arguing that as
the size of a dataset increases, the performance of a model trained on that dataset will …

Large language models have demonstrated great potential to assist programmers in
generating code. For such human-AI pair programming scenarios, we empirically …

A core step in production model research and development involves the offline evaluation of
a system before production deployment. Traditional offline evaluation of search …

Recommender systems have become the dominant means of curating cultural content,
significantly influencing the nature of individual cultural experience. While the majority of …

Although originally developed to evaluate sets of items, recall is often used to evaluate
rankings of items, including those produced by recommender, retrieval, and other machine …

Across a variety of ranking tasks, researchers use reciprocal rank to measure the
effectiveness for users interested in exactly one relevant item. Despite its widespread use …

Text-to-Image (TTI) systems often support people during ideation, the early stages of a
creative process when exposure to a broad set of relevant or partially relevant images can …

Many information access systems operationalize their results in terms of rankings, which are
then displayed to users in various ranking layouts such as linear lists or grids. User …

Designers of online search and recommendation services often need to develop metrics to
assess system performance. This tutorial focuses on mixed methods approaches to …