Overview of the clef-2023 longeval lab on longitudinal evaluation of model performance
We describe the first edition of the LongEval CLEF 2023 shared task. This lab evaluates the
temporal persistence of Information Retrieval (IR) systems and Text Classifiers. Task 1 …
temporal persistence of Information Retrieval (IR) systems and Text Classifiers. Task 1 …
On Fuhr's guideline for IR evaluation
T Sakai - ACM SIGIR Forum, 2021 - dl.acm.org
In the December 2017 issue of SIGIR Forum, Fuhr presented ten" Thou Shalt Not" s (ie,
warnings against bad practices) for IR experimenters. While his article provides a lot of good …
warnings against bad practices) for IR experimenters. While his article provides a lot of good …
Deep neural network regression for normalized digital surface model generation with Sentinel-2 imagery
In recent history, normalized digital surface models (nDSMs) have been constantly gaining
importance as a means to solve large-scale geographic problems. High-resolution surface …
importance as a means to solve large-scale geographic problems. High-resolution surface …
Feature Attention Distillation Defense for Backdoor Attack in Artificial Neural Network-Based Electricity Theft Detection
Artificial neural networks (ANNs) have been widely used for tasks like electricity theft
detection (ETD) in smart meters. However, due to the subtle mechanisms and inherent …
detection (ETD) in smart meters. However, due to the subtle mechanisms and inherent …
Modelling the interrelationships between potential risk factors and childhood Co-morbidity of Malaria, Anaemia, and stunting in children less than five years in Burundi
Background Anaemia, malaria, and stunting remain health problems, especially in children
younger than five years, and those conditions are linked to morbidity and mortality. The main …
younger than five years, and those conditions are linked to morbidity and mortality. The main …
Detecting significant differences between information retrieval systems via generalized linear models
Being able to compare Information Retrieval (IR) systems correctly is pivotal to improving
their quality. Among the most popular tools for statistical significance testing, we list t-test …
their quality. Among the most popular tools for statistical significance testing, we list t-test …
Towards the evaluation of information retrieval systems on evolving datasets with pivot systems
Abstract Evaluation of information retrieval systems follows the Cranfield paradigm, where
the evaluation of several IR systems relies on a common evaluation environment (test …
the evaluation of several IR systems relies on a common evaluation environment (test …
Towards meaningful statements in IR evaluation: Map** evaluation measures to interval scales
Information Retrieval (IR) is a discipline deeply rooted in evaluation since its inception.
Indeed, experimentally measuring and statistically validating the performance of IR systems …
Indeed, experimentally measuring and statistically validating the performance of IR systems …
Exploratory visualization tool for the continuous evaluation of information retrieval systems
This paper introduces a novel visualization tool that facilitates the exploratory analysis of
continuous evaluation for information retrieval systems. We base our analysis on score …
continuous evaluation for information retrieval systems. We base our analysis on score …
[PDF][PDF] Fairness-based evaluation of conversational search: A pilot study
T Sakai - Proceedings of EVIA, 2023 - repository.nii.ac.jp
ABSTRACT NTCIR-17 introduced the FairWeb-1 task, which evaluated web page rankings
in terms of both relevance and group fairness. The present study shows how their evaluation …
in terms of both relevance and group fairness. The present study shows how their evaluation …