Artificial neural networks (ANNs) inspired by biology are beginning to be widely used to
model behavioural and neural data, an approach we call 'neuroconnectionism'. ANNs have …

In this work, we survey a breadth of literature that has revealed the limitations of
predominant practices for dataset collection and use in the field of machine learning. We …

Human variation in labeling is often considered noise. Annotation projects for machine
learning (ML) aim at minimizing human label variation, with the assumption to maximize …

There is a tendency across different subfields in AI to valorize a small collection of influential
benchmarks. These benchmarks operate as stand-ins for a range of anointed common …

Benchmark datasets play a central role in the organization of machine learning research.
They coordinate researchers around shared research problems and serve as a measure of …

More than one hundred benchmarks have been developed to test the commonsense
knowledge and commonsense reasoning abilities of artificial intelligence (AI) systems …

Forming a reliable judgement of a machine learning (ML) model's appropriateness for an
application ecosystem is critical for its responsible use, and requires considering a broad …

Much of the recent discourse within the ML community has been centered around Large
Language Models (LLMs), their functionality and potential--yet not only do we not have a …

Leaderboards are widely used in NLP and push the field forward. While leaderboards are a
straightforward ranking of NLP models, this simplicity can mask nuances in evaluation items …

Questions regarding implicitness, ambiguity and underspecification are crucial for
understanding the task validity and ethical concerns of multimodal image+ text systems, yet …