With the emergence of various information access systems exhibiting increasing complexity,
there is a critical need for sound and scalable means of automatic evaluation. To address …

Prior research on exposure fairness in the context of recommender systems has focused
mostly on disparities in the exposure of individual or groups of items to individual users of …

Current practice for evaluating recommender systems typically focuses on point estimates of
user-oriented effectiveness metrics or business metrics, sometimes combined with …

Many current effectiveness measures incorporate simplifying assumptions about user
behavior. These assumptions prevent the measures from reflecting aspects of the search …

Understanding user browsing behavior in web search is key to improving web search
effectiveness. Many click models have been proposed to explain or predict user clicks on …

There is great interest in producing effectiveness measures that model user behavior in
order to better model the utility of a system to its users. These measures are often formulated …

The amounts of available Semantic Web (SW) data (including Linked Open Data) constantly
increases. Users would like to browse and explore effectively such information spaces …

Many web systems rank and present a list of items to users, from recommender systems to
search and advertising. An important problem in practice is to evaluate new ranking policies …

Information retrieval systems aim to help users satisfy information needs. We argue that the
goal of the person using the system, and the pattern of behavior that they exhibit as they …

Traditional editorial effectiveness measures, such as nDCG, remain standard for Web
search evaluation. Unfortunately, these traditional measures can inappropriately reward …