Continual learning of large language models: A comprehensive survey

H Shi, Z Xu, H Wang, W Qin, W Wang, Y Wang… - arxiv preprint arxiv …, 2024 - arxiv.org
The recent success of large language models (LLMs) trained on static, pre-collected,
general datasets has sparked numerous research directions and applications. One such …

Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning

W Ren, X Li, L Wang, T Zhao, W Qin - arxiv preprint arxiv:2402.18865, 2024 - arxiv.org
Existing research has shown that large language models (LLMs) exhibit remarkable
performance in language understanding and generation. However, when LLMs are …

The State and Fate of Summarization Datasets

N Dahan, G Stanovsky - arxiv preprint arxiv:2411.04585, 2024 - arxiv.org
Automatic summarization has consistently attracted attention, due to its versatility and wide
application in various downstream tasks. Despite its popularity, we find that annotation …

Fine-tuning the SwissBERT Encoder Model for Embedding Sentences and Documents

J Grosjean, J Vamvas - arxiv preprint arxiv:2405.07513, 2024 - arxiv.org
Encoder models trained for the embedding of sentences or short documents have proven
useful for tasks such as semantic search and topic modeling. In this paper, we present a …

[PDF][PDF] Tracing Linguistic Footprints of ChatGPT Across Tasks, Domains and Personas in English and German

A Shaitarova, N Bauer, J Vamvas… - … of the 9th edition of the …, 2024 - aclanthology.org
Large language models like ChatGPT can be used to generate seemingly human-like text.
However, it is still not well understood how their output differs from text written by humans …

Towards a Unified Framework for Aspect-based Multi-document Text Summarization

D Aumiller - 2024 - archiv.ub.uni-heidelberg.de
For a growing number of knowledge workers, the rapid ingestion of textual information is
crucial for their daily tasks. Confronted with expansive bodies of text, the fastest way to glean …