Abstract Evaluation practices in natural language generation (NLG) have many known flaws,
but improved evaluation approaches are rarely widely adopted. This issue has become …

Machine-learning (ML) methods have gained prominence in the quantitative sciences.
However, there are many known methodological pitfalls, including data leakage, in ML …

The ability to generalize well is one of the primary desiderata for models of natural language
processing (NLP), but what 'good generalization'entails and how it should be evaluated is …

Pretrained Language Models (LMs) have demonstrated ability to perform numerical
reasoning by extrapolating from a few examples in few-shot settings. However, the extent to …

We present the results and main findings of SemEval-2020 Task 12 on Multilingual
Offensive Language Identification in Social Media (OffensEval 2020). The task involves …

Research in natural language processing proceeds, in part, by demonstrating that new
models achieve superior performance (eg, accuracy) on held-out test data, compared to …

Large pre-trained language models (PTLMs) have been shown to carry biases towards
different social groups which leads to the reproduction of stereotypical and toxic content by …

Moderation is crucial to promoting healthy on-line discussions. Although severaltoxicity'
detection datasets and models have been published, most of them ignore the context of the …

We introduce MULTI-EURLEX, a new multilingual dataset for topic classification of legal
documents. The dataset comprises 65k European Union (EU) laws, officially translated in 23 …

Abstract Out-of-distribution (OOD) testing is increasingly popular for evaluating a machine
learning system's ability to generalize beyond the biases of a training set. OOD benchmarks …