Citations are the cornerstone of knowledge propagation and the primary means of
assessing the quality of research, as well as directing investments in science. Science is …

Reproducibility is a key requirement for scientific progress. It allows the reproduction of the
works of others, and, as a consequence, to fully trust the reported claims and results. In this …

Science is facing a so-called reproducibility crisis, where researchers struggle to repeat
experiments and to get the same or comparable results. This represents a fundamental …

Context Independent validation of published scientific results through study replication is a
pre-condition for accepting the validity of such results. In computation research, full …

Recently, the ACM created a policy on Artifact Review and Badging, which presents a
framework to help SIGs recognize repeatability, replicability and reproducibility in published …

In computing, research findings are often anecdotally faulted for not being reproducible.
Numerous empirical studies have analyzed the reproducibility of a variety of research. Our …

The Conversational Search (CS) paradigm allows for an intuitive interaction between the
user and the system through natural language sentences and it is increasingly being …

In the research of video retrieval systems, comparative assessments during dedicated
retrieval competitions provide priceless insights into the performance of individual systems …

In this work we introduce repro_eval-a tool for reactive reproducibility studies of system-
oriented Information Retrieval (IR) experiments. The corresponding Python package …

Reproducibility has become increasingly important for many research areas, among those
IR is not an exception and has started to be concerned with reproducibility and its impact on …