In current NLP research, large-scale language models and their abilities are widely being
discussed. Some recent works have also found notable failures of these models. Often these …

Do all instances need inference through the big models for a correct prediction? Perhaps
not; some instances are easy and can be answered correctly by even small capacity models …

Human evaluation is viewed as a reliable evaluation method for NLG which is expensive
and time-consuming. In order to save labor and costs, researchers usually perform human …

It's better to say" I can't answer" than to answer incorrectly. This selective prediction ability is
crucial for NLP systems to be reliably deployed in real-world applications. Prior work has …

Recent state-of-the-art open-domain QA models are typically based on a two stage retriever-
reader approach in which the retriever first finds the relevant knowledge/passages and the …

Pretrained natural language processing (NLP) models have achieved high overall
performance, but they still make systematic errors. Instead of manual error analysis …

Manually annotated datasets are crucial for training and evaluating Natural Language
Processing models. However, recent work has discovered that even widely-used benchmark …

While pretrained language models have exhibited impressive generalization capabilities,
they still behave unpredictably under certain domain shifts. In particular, a model may learn …

In recent years, data-driven and machine learning-based natural language processing
(NLP) technologies have effectively addressed various challenges. To further enhance the …

The development of state-of-the-art systems in different applied areas of machine learning
(ML) is driven by benchmarks, which have shaped the paradigm of evaluating …