We present a self-supervised learning framework, COCO-LM, that pretrains Language
Models by COrrecting and COntrasting corrupted text sequences. Following ELECTRA-style …

Abstract Open-domain Question Answering models that directly leverage question-answer
(QA) pairs, such as closed-book QA (CBQA) models and QA-pair retrievers, show promise in …

Long-form question answering (LFQA) aims to generate a paragraph-length answer for a
given question. While current work on LFQA using large pre-trained model for generation …

Salient Span Masking (SSM) has shown itself to be an effective strategy to improve closed-
book question answering performance. SSM extends general masked language model …

The Open-Domain Question Answering (ODQA) task involves retrieving and subsequently
generating answers from fine-grained relevant passages within a database. Current systems …

Current NLP models are predominantly trained through a two-stage" pre-train then fine-tune"
pipeline. Prior work has shown that inserting an intermediate pre-training stage, using …

Socially intelligent systems such as home robots should be able to perceive emotions and
social behaviors. Affect recognition datasets have limited labeled data, and existing large …

Transformer-based language models (LMs) have been shown to perform question
answering (QA) competitively even when removing context and using only questions as …

Abstract Language models such as Chat-GPT generate new answers from the data they
learn, improving work efficiency and generating ideas. On the other hand, incorrect answers …

論文抄録 ニューラルネットワークを用いた質問応答では, 機械読解やオープン検索質問応答などが
研究され, **年はクローズドブック質問応答の研究が進められている. 本研究では …