Continual learning for large language models: A survey

T Wu, L Luo, YF Li, S Pan, TT Vu, G Haffari - arxiv preprint arxiv …, 2024 - arxiv.org
Large language models (LLMs) are not amenable to frequent re-training, due to high
training costs arising from their massive scale. However, updates are necessary to endow …

Recent advances of foundation language models-based continual learning: A survey

Y Yang, J Zhou, X Ding, T Huai, S Liu, Q Chen… - ACM Computing …, 2025 - dl.acm.org
Recently, foundation language models (LMs) have marked significant achievements in the
domains of natural language processing and computer vision. Unlike traditional neural …

Self-Updatable Large Language Models with Parameter Integration

Y Wang, X Liu, X Chen, S O'Brien, J Wu… - arxiv preprint arxiv …, 2024 - arxiv.org
Despite significant advancements in large language models (LLMs), the rapid and frequent
integration of small-scale experiences, such as interactions with surrounding objects …

Towards lifespan cognitive systems

Y Wang, C Han, T Wu, X He, W Zhou, N Sadeq… - arxiv preprint arxiv …, 2024 - arxiv.org
Building a human-like system that continuously interacts with complex environments--
whether simulated digital worlds or human society--presents several key challenges. Central …

Multi-LoRA continual learning based instruction tuning framework for universal information extraction

Y **, J Liu, S Chen - Knowledge-Based Systems, 2025 - Elsevier
Universal information extraction (Universal IE) aims to develop one model capable of
solving multiple IE target tasks. Previous works have enhanced extraction performance of …

MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning

Y Chen, S Wang, Z Lin, Z Qin, Y Zhang, T Ding… - arxiv preprint arxiv …, 2024 - arxiv.org
Recently, large language models (LLMs) have demonstrated remarkable capabilities in a
wide range of tasks. Typically, an LLM is pre-trained on large corpora and subsequently fine …

[PDF][PDF] Task-incremental learning on long text sequences

N Graziuso, A Zugarini, S Melacci - … of the Tenth Italian Conference on …, 2024 - ceur-ws.org
The extraordinary results achieved by Large Language Models are paired with issues that
are critical in real-world applications. The costs of inference and, in particular, training are …

Evaluation on parameter-efficient continual instruction tuning of large language models

TX Ren, XS Song, JY Shi, JS Deng… - … Conference on Control …, 2024 - spiedigitallibrary.org
Recent years have witnessed a spurt in large language models for its strong generalization
performance, many continual instruction tuning methods based on parameter-efficient tuning …

Mitigating Catastrophic Forgetting in Task-Incremental Learning for Large Language Models

M Shakya - 2024 - doria.fi
Large Language Models have shown the capability to perform well on various tasks. To
achieve this, the models are trained on vast amounts of data. However, this knowledge can …