Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual
explanations. These explanations are used downstream in three ways: as data …

We present Belebele, a multiple-choice machine reading comprehension (MRC) dataset
spanning 122 language variants. Significantly expanding the language coverage of natural …

Evaluation for many natural language understanding (NLU) tasks is broken: Unreliable and
biased systems score so highly on standard benchmarks that there is little room for …

A recurring challenge of crowdsourcing NLP datasets at scale is that human writers often
rely on repetitive patterns when crafting examples, leading to a lack of linguistic diversity. We …

Natural language inference (NLI) has been widely used as a task to train and evaluate
models for language understanding. However, the ability of NLI models to perform …

The general format of natural language inference (NLI) makes it tempting to be used for zero-
shot text classification by casting any target label into a sentence of hypothesis and verifying …

In Dynamic Adversarial Data Collection (DADC), human annotators are tasked with finding
examples that models struggle to predict correctly. Models trained on DADC-collected …

To create models that are robust across a wide range of test inputs, training datasets should
include diverse examples that span numerous phenomena. Dynamic adversarial data …

We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs
collected through a fun multi-player game. Gamification encourages adversarial examples …

With the considerable achievements of data-hungry deep learning methods in natural
language processing tasks, a great amount of effort has been devoted to develop more …