Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual
explanations. These explanations are used downstream in three ways: as data …

Human variation in labeling is often considered noise. Annotation projects for machine
learning (ML) aim at minimizing human label variation, with the assumption to maximize …

Natural language generation has witnessed significant advancements due to the training of
large language models on vast internet-scale datasets. Despite these advancements, there …

Abstract Many tasks in Natural Language Processing (NLP) and Computer Vision (CV) offer
evidence that humans disagree, from objective tasks such as part-of-speech tagging to more …

Human feedback plays a central role in the alignment of Large Language Models (LLMs).
However, open questions remain about the methods (how), domains (where), people (who) …

Recognizing if LLM output can be grounded in evidence is central to many tasks in NLP:
retrieval-augmented generation, summarization, document-grounded dialogue, and more …

Ambiguity is an intrinsic feature of natural language. Managing ambiguity is a key part of
human language understanding, allowing us to anticipate misunderstanding as …

Human feedback is central to the alignment of Large Language Models (LLMs). However,
open questions remain about the methods (how), domains (where), people (who) and …

Humans produce and consume language in a particular cultural context, which includes
knowledge about specific norms and practices. A listener's awareness of the cultural context …

We investigate how disagreement in natural language inference (NLI) annotation arises. We
developed a taxonomy of disagreement sources with 10 categories spanning 3 high-level …