Few-shot learning (FSL) has emerged as an effective learning method and shows great
potential. Despite the recent creative works in tackling FSL tasks, learning valid information …

Despite the recent success of multi-task learning and transfer learning for natural language
processing (NLP), few works have systematically studied the effect of scaling up the number …

Existing language models (LMs) predict tokens with a softmax over a finite vocabulary,
which can make it difficult to predict rare tokens or phrases. We introduce NPM, the first …

Pretrained models are the standard starting point for training. This approach consistently
outperforms the use of a random initialization. However, pretraining is a costly endeavour …

The information in tables can be an important complement to text, making table-based
question answering (QA) systems of great value. The intrinsic complexity of handling tables …

Dense retrievers for open-domain question answering (ODQA) have been shown to achieve
impressive performance by training on large datasets of question-passage pairs. In this work …

Large pretrained language models (LMs) have become the central building block of many
NLP applications. Training these models requires ever more computational resources and …

Skill Extraction (SE) is an important and widely-studied task useful to gain insights into labor
market dynamics. However, there is a lacuna of datasets and annotation guidelines; …

Models pre-trained with a language modeling objective possess ample world knowledge
and language skills, but are known to struggle in tasks that require reasoning. In this work …

Question Answering (QA) is a longstanding challenge in natural language processing.
Existing QA works mostly focus on specific question types, knowledge domains, or …