Evaluating recommender systems: survey and framework

E Zangerle, C Bauer - ACM computing surveys, 2022 - dl.acm.org
The comprehensive evaluation of the performance of a recommender system is a complex
endeavor: many facets need to be considered in configuring an adequate and effective …

A troubling analysis of reproducibility and progress in recommender systems research

M Ferrari Dacrema, S Boglio, P Cremonesi… - ACM Transactions on …, 2021 - dl.acm.org
The design of algorithms that generate personalized ranked item lists is a central topic of
research in the field of recommender systems. In the past few years, in particular …

Building human values into recommender systems: An interdisciplinary synthesis

J Stray, A Halevy, P Assar, D Hadfield-Menell… - ACM Transactions on …, 2024 - dl.acm.org
Recommender systems are the algorithms which select, filter, and personalize content
across many of the world's largest platforms and apps. As such, their positive and negative …

Exploring data splitting strategies for the evaluation of recommendation models

Z Meng, R McCreadie, C Macdonald… - Proceedings of the 14th …, 2020 - dl.acm.org
Effective methodologies for evaluating recommender systems are critical, so that different
systems can be compared in a sound manner. A commonly overlooked aspect of evaluating …

Quality metrics in recommender systems: Do we calculate metrics consistently?

YM Tamm, R Damdinov, A Vasilev - … of the 15th ACM conference on …, 2021 - dl.acm.org
Offline evaluation is a popular approach to determine the best algorithm in terms of the
chosen quality metric. However, if the chosen metric calculates something unexpected, this …

A Revisiting Study of Appropriate Offline Evaluation for Top-N Recommendation Algorithms

WX Zhao, Z Lin, Z Feng, P Wang, JR Wen - ACM Transactions on …, 2022 - dl.acm.org
In recommender systems, top-N recommendation is an important task with implicit feedback
data. Although the recent success of deep learning largely pushes forward the research on …

A critical study on data leakage in recommender system offline evaluation

Y Ji, A Sun, J Zhang, C Li - ACM Transactions on Information Systems, 2023 - dl.acm.org
Recommender models are hard to evaluate, particularly under offline setting. In this article,
we provide a comprehensive and critical analysis of the data leakage issue in recommender …

Offline recommender system evaluation: Challenges and new directions

P Castells, A Moffat - AI magazine, 2022 - ojs.aaai.org
Offline evaluation is an essential complement to online experiments in the selection,
improvement, tuning, and deployment of recommender systems. Offline methodologies for …

Exploring artist gender bias in music recommendation

D Shakespeare, L Porcaro, E Gómez… - arxiv preprint arxiv …, 2020 - arxiv.org
Music Recommender Systems (mRS) are designed to give personalised and meaningful
recommendations of items (ie songs, playlists or artists) to a user base, thereby reflecting …

On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation

O Jeunen, I Potapov, A Ustimenko - Proceedings of the 30th ACM …, 2024 - dl.acm.org
Approaches to recommendation are typically evaluated in one of two ways:(1) via a
(simulated) online experiment, often seen as the gold standard, or (2) via some offline …