The number of publications on deep learning for cancer diagnostics is rapidly increasing,
and systems are frequently claimed to perform comparable with or better than clinicians …

Many subfields of machine learning share a common stumbling block: evaluation. Advances
in machine learning often evaporate under closer scrutiny or turn out to be less widely …

The capabilities of a new class of tools, colloquially known as generative artificial
intelligence (AI), is a topic of much debate. One prominent application thus far is the …

Generative AI systems across modalities, ranging from text, image, audio, and video, have
broad social impacts, but there exists no official standard for means of evaluating those …

Unsupervised localization and segmentation are long-standing computer vision challenges
that involve decomposing an image into semantically-meaningful segments without any …

Existing public face image datasets are strongly biased toward Caucasian faces, and other
races (eg, Latino) are significantly underrepresented. The models trained from such datasets …

We study how robust current ImageNet models are to distribution shifts arising from natural
variations in datasets. Most research on robustness focuses on synthetic image …

Data is a crucial component of machine learning. The field is reliant on data to train, validate,
and test models. With increased technical capabilities, machine learning research has …

In this paper we investigate problematic practices and consequences of large scale vision
datasets (LSVDs). We examine broad issues such as the question of consent and justice as …

Datasets that power machine learning are often used, shared, and reused with little visibility
into the processes of deliberation that led to their creation. As artificial intelligence systems …