A survey on long text modeling with transformers

Z Dong, T Tang, L Li, WX Zhao - arxiv preprint arxiv:2302.14502, 2023 - arxiv.org
Modeling long texts has been an essential technique in the field of natural language
processing (NLP). With the ever-growing number of long documents, it is important to …

Summ^ n: A multi-stage summarization framework for long input dialogues and documents

Y Zhang, A Ni, Z Mao, CH Wu, C Zhu, B Deb… - arxiv preprint arxiv …, 2021 - arxiv.org
Text summarization helps readers capture salient information from documents, news,
interviews, and meetings. However, most state-of-the-art pretrained language models (LM) …

Efficient memory-enhanced transformer for long-document summarization in low-resource regimes

G Moro, L Ragazzi, L Valgimigli, G Frisoni, C Sartori… - Sensors, 2023 - mdpi.com
Long document summarization poses obstacles to current generative transformer-based
models because of the broad context to process and understand. Indeed, detecting long …

Single-Document Abstractive Text Summarization: A Systematic Literature Review

A Rao, S Aithal, S Singh - ACM Computing Surveys, 2024 - dl.acm.org
Abstractive text summarization is a task in natural language processing that automatically
generates the summary from the source document in a human-written form with minimal loss …

Toward unifying text segmentation and long document summarization

S Cho, K Song, X Wang, F Liu, D Yu - arxiv preprint arxiv:2210.16422, 2022 - arxiv.org
Text segmentation is important for signaling a document's structure. Without segmenting a
long document into topically coherent sections, it is difficult for readers to comprehend the …

Carburacy: summarization models tuning and comparison in eco-sustainable regimes with a novel carbon-aware accuracy

G Moro, L Ragazzi, L Valgimigli - … of the AAAI Conference on Artificial …, 2023 - ojs.aaai.org
Generative transformer-based models have reached cutting-edge performance in long
document summarization. Nevertheless, this task is witnessing a paradigm shift in …

How far are we from robust long abstractive summarization?

HY Koh, J Ju, H Zhang, M Liu, S Pan - arxiv preprint arxiv:2210.16732, 2022 - arxiv.org
Abstractive summarization has made tremendous progress in recent years. In this work, we
perform fine-grained human annotations to evaluate long document abstractive …

[HTML][HTML] Align-then-abstract representation learning for low-resource summarization

G Moro, L Ragazzi - Neurocomputing, 2023 - Elsevier
Generative transformer-based models have achieved state-of-the-art performance in text
summarization. Nevertheless, they still struggle in real-world scenarios with long documents …

Leveraging locality in abstractive text summarization

Y Liu, A Ni, L Nan, B Deb, C Zhu, AH Awadallah… - arxiv preprint arxiv …, 2022 - arxiv.org
Neural attention models have achieved significant improvements on many natural language
processing tasks. However, the quadratic memory complexity of the self-attention module …

SNaC: Coherence error detection for narrative summarization

T Goyal, JJ Li, G Durrett - arxiv preprint arxiv:2205.09641, 2022 - arxiv.org
Progress in summarizing long texts is inhibited by the lack of appropriate evaluation
frameworks. When a long summary must be produced to appropriately cover the facets of …