Ranking has been extensively studied in information retrieval, machine learning and
statistics. A central problem in ranking is to design a ranking measure for evaluation of …

Information retrieval (IR) researchers commonly use three tests of statistical significance: the
Student's paired t-test, the Wilcoxon signed rank test, and the sign test. Other researchers …

Use of test collections and evaluation measures to assess the effectiveness of information
retrieval systems has its origins in work dating back to the early 1950s. Across the nearly 60 …

We conducted a systematic review of 840 SIGIR full papers and 215 TOIS papers published
between 2006 and 2015. The original objective of the study was to identify IR effectiveness …

Ranking in information retrieval has been traditionally approached as a pursuit of relevant
information, under the assumption that the users' information needs are unambiguously …

The evaluation of recommender systems is an area with unsolved questions at several
levels. Choosing the appropriate evaluation metric is one of such important issues. Ranking …

Many current effectiveness measures incorporate simplifying assumptions about user
behavior. These assumptions prevent the measures from reflecting aspects of the search …

In this report we discuss the experiments we conducted at the University of Wolverhampton
for the Microblog Track at TREC-2011. As this was the first time we participated in TREC and …

In multi-class text classification, the performance (effectiveness) of a classifier is usually
measured by micro-averaged and macro-averaged F 1 scores. However, the scores …

Statistical significance testing is widely accepted as a means to assess how well a difference
in effectiveness reflects an actual difference between systems, as opposed to random noise …