Continual learning for large language models: A survey
Large language models (LLMs) are not amenable to frequent re-training, due to high
training costs arising from their massive scale. However, updates are necessary to endow …
training costs arising from their massive scale. However, updates are necessary to endow …
Recent advances of foundation language models-based continual learning: A survey
Recently, foundation language models (LMs) have marked significant achievements in the
domains of natural language processing and computer vision. Unlike traditional neural …
domains of natural language processing and computer vision. Unlike traditional neural …
Self-Updatable Large Language Models with Parameter Integration
Despite significant advancements in large language models (LLMs), the rapid and frequent
integration of small-scale experiences, such as interactions with surrounding objects …
integration of small-scale experiences, such as interactions with surrounding objects …
Towards lifespan cognitive systems
Building a human-like system that continuously interacts with complex environments--
whether simulated digital worlds or human society--presents several key challenges. Central …
whether simulated digital worlds or human society--presents several key challenges. Central …
Multi-LoRA continual learning based instruction tuning framework for universal information extraction
Y **, J Liu, S Chen - Knowledge-Based Systems, 2025 - Elsevier
Universal information extraction (Universal IE) aims to develop one model capable of
solving multiple IE target tasks. Previous works have enhanced extraction performance of …
solving multiple IE target tasks. Previous works have enhanced extraction performance of …
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning
Y Chen, S Wang, Z Lin, Z Qin, Y Zhang, T Ding… - arxiv preprint arxiv …, 2024 - arxiv.org
Recently, large language models (LLMs) have demonstrated remarkable capabilities in a
wide range of tasks. Typically, an LLM is pre-trained on large corpora and subsequently fine …
wide range of tasks. Typically, an LLM is pre-trained on large corpora and subsequently fine …
[PDF][PDF] Task-incremental learning on long text sequences
The extraordinary results achieved by Large Language Models are paired with issues that
are critical in real-world applications. The costs of inference and, in particular, training are …
are critical in real-world applications. The costs of inference and, in particular, training are …
Evaluation on parameter-efficient continual instruction tuning of large language models
TX Ren, XS Song, JY Shi, JS Deng… - … Conference on Control …, 2024 - spiedigitallibrary.org
Recent years have witnessed a spurt in large language models for its strong generalization
performance, many continual instruction tuning methods based on parameter-efficient tuning …
performance, many continual instruction tuning methods based on parameter-efficient tuning …
Mitigating Catastrophic Forgetting in Task-Incremental Learning for Large Language Models
M Shakya - 2024 - doria.fi
Large Language Models have shown the capability to perform well on various tasks. To
achieve this, the models are trained on vast amounts of data. However, this knowledge can …
achieve this, the models are trained on vast amounts of data. However, this knowledge can …