Pre-trained models for natural language processing: A survey

X Qiu, T Sun, Y Xu, Y Shao, N Dai, X Huang - Science China …, 2020 - Springer
Recently, the emergence of pre-trained models (PTMs) has brought natural language
processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs …

Deep reinforcement and transfer learning for abstractive text summarization: A review

A Alomari, N Idris, AQM Sabri, I Alsmadi - Computer Speech & Language, 2022 - Elsevier
Abstract Automatic Text Summarization (ATS) is an important area in Natural Language
Processing (NLP) with the goal of shortening a long text into a more compact version by …

Bartscore: Evaluating generated text as text generation

W Yuan, G Neubig, P Liu - Advances in Neural Information …, 2021 - proceedings.neurips.cc
A wide variety of NLP applications, such as machine translation, summarization, and dialog,
involve text generation. One major challenge for these applications is how to evaluate …

Towards a unified multi-dimensional evaluator for text generation

M Zhong, Y Liu, D Yin, Y Mao, Y Jiao, P Liu… - arxiv preprint arxiv …, 2022 - arxiv.org
Multi-dimensional evaluation is the dominant paradigm for human evaluation in Natural
Language Generation (NLG), ie, evaluating the generated text from multiple explainable …

Pegasus: Pre-training with extracted gap-sentences for abstractive summarization

J Zhang, Y Zhao, M Saleh, P Liu - … conference on machine …, 2020 - proceedings.mlr.press
Recent work pre-training Transformers with self-supervised objectives on large text corpora
has shown great success when fine-tuned on downstream NLP tasks including text …

Extractive summarization as text matching

M Zhong, P Liu, Y Chen, D Wang, X Qiu… - arxiv preprint arxiv …, 2020 - arxiv.org
This paper creates a paradigm shift with regard to the way we build neural extractive
summarization systems. Instead of following the commonly used framework of extracting …

QMSum: A new benchmark for query-based multi-domain meeting summarization

M Zhong, D Yin, T Yu, A Zaidi, M Mutuma, R Jha… - arxiv preprint arxiv …, 2021 - arxiv.org
Meetings are a key component of human collaboration. As increasing numbers of meetings
are recorded and transcribed, meeting summaries have become essential to remind those …

Heterogeneous graph neural networks for extractive document summarization

D Wang, P Liu, Y Zheng, X Qiu, X Huang - arxiv preprint arxiv:2004.12393, 2020 - arxiv.org
As a crucial step in extractive document summarization, learning cross-sentence relations
has been explored by a plethora of approaches. An intuitive way is to put them in the graph …

Discourse-aware neural extractive text summarization

J Xu, Z Gan, Y Cheng, J Liu - arxiv preprint arxiv:1910.14142, 2019 - arxiv.org
Recently BERT has been adopted for document encoding in state-of-the-art text
summarization models. However, sentence-based extractive models often result in …

Re-evaluating evaluation in text summarization

M Bhandari, P Gour, A Ashfaq, P Liu… - arxiv preprint arxiv …, 2020 - arxiv.org
Automated evaluation metrics as a stand-in for manual evaluation are an essential part of
the development of text-generation tasks such as text summarization. However, while the …