Challenges, experiments, and computational solutions in peer review Page 1 76
COMMUNICATIONS OF THE ACM | JUNE 2022 | VOL. 65 | NO. 6 review articles IMA GER YB Y …

In peer mechanisms, the competitors for a prize also determine who wins. Each competitor
may be asked to rank, grade, or nominate peers for the prize. Since the prize can be …

Algorithms and machine learning models, including the decisions made by these models,
are becoming ubiquitous in our daily life, including hiring. We make no value judgment …

We consider three important challenges in conference peer review:(i) reviewers maliciously
attempting to get assigned to certain papers to provide positive reviews, possibly as part of …

The number of scientific articles produced every year is growing rapidly. Providing quality
control over them is crucial for scientists and, ultimately, for the public good. In modern …

Is it possible to reliably evaluate the quality of peer reviews? We study this question driven
by two primary motivations--incentivizing high-quality reviewing using assessed quality of …

Quantifying systematic disparities in numerical quantities such as employment rates and
wages between population subgroups provides compelling evidence for the existence of …

Auto-evaluation is crucial for assessing response quality and offering feedback for model
development. Recent studies have explored training large language models (LLMs) as …

We consider the issue of biases in scholarly research, specifically, in peer review. There is a
long standing debate on whether exposing author identities to reviewers induces biases …

Strategic behavior is a fundamental problem in a variety of real-world applications that
require some form of peer assessment, such as peer grading of homeworks, grant proposal …