The field of natural language processing has seen impressive progress in recent years, with
neural network models replacing many of the traditional systems. A plethora of new models …

Deep learning has recently come to dominate computational linguistics, leading to claims of
human-level performance in a range of language processing tasks. Like much previous …

Abstract Many tasks in Natural Language Processing (NLP) and Computer Vision (CV) offer
evidence that humans disagree, from objective tasks such as part-of-speech tagging to more …

While language models are powerful and versatile, they often fail to address highly complex
problems. This is because solving complex problems requires deliberate thinking, which has …

Large language models (LLMs) have achieved remarkable performance on a variety of
natural language understanding tasks. However, existing benchmarks are inadequate in …

We analyze human's disagreements about the validity of natural language inferences. We
show that, very often, disagreements are not dismissible as annotation “noise”, but rather …

Evaluation for many natural language understanding (NLU) tasks is broken: Unreliable and
biased systems score so highly on standard benchmarks that there is little room for …

Natural language inference (NLI) is the task of determining if a natural language hypothesis
can be inferred from a given premise in a justifiable manner. NLI was proposed as a …

The recently increased focus on misinformation has stimulated research in fact checking, the
task of assessing the truthfulness of a claim. Research in automating this task has been …

This paper presents the task on the evaluation of Compositional Distributional Semantics
Models on full sentences organized for the first time within SemEval-2014. Participation was …