Continual learning of large language models: A comprehensive survey
The recent success of large language models (LLMs) trained on static, pre-collected,
general datasets has sparked numerous research directions and applications. One such …
general datasets has sparked numerous research directions and applications. One such …
Recent advances of foundation language models-based continual learning: A survey
Recently, foundation language models (LMs) have marked significant achievements in the
domains of natural language processing (NLP) and computer vision (CV). Unlike traditional …
domains of natural language processing (NLP) and computer vision (CV). Unlike traditional …
A survey on model moerging: Recycling and routing among specialized experts for collaborative learning
The availability of performant pre-trained models has led to a proliferation of fine-tuned
expert models that are specialized to a particular domain or task. Model MoErging methods …
expert models that are specialized to a particular domain or task. Model MoErging methods …
Select and distill: Selective dual-teacher knowledge transfer for continual learning on vision-language models
Large-scale vision-language models (VLMs) have shown a strong zero-shot generalization
capability on unseen-domain data. However, adapting pre-trained VLMs to a sequence of …
capability on unseen-domain data. However, adapting pre-trained VLMs to a sequence of …
A Practitioner's Guide to Continual Multimodal Pretraining
Multimodal foundation models serve numerous applications at the intersection of vision and
language. Still, despite being pretrained on extensive data, they become outdated over time …
language. Still, despite being pretrained on extensive data, they become outdated over time …
Clip with generative latent replay: a strong baseline for incremental learning
With the emergence of Transformers and Vision-Language Models (VLMs) such as CLIP,
fine-tuning large pre-trained models has recently become a prevalent strategy in Continual …
fine-tuning large pre-trained models has recently become a prevalent strategy in Continual …
Theory on mixture-of-experts in continual learning
Continual learning (CL) has garnered significant attention because of its ability to adapt to
new tasks that arrive over time. Catastrophic forgetting (of old tasks) has been identified as a …
new tasks that arrive over time. Catastrophic forgetting (of old tasks) has been identified as a …
Llms can evolve continually on modality for x-modal reasoning
Multimodal Large Language Models (MLLMs) have gained significant attention due to their
impressive capabilities in multimodal understanding. However, existing methods rely heavily …
impressive capabilities in multimodal understanding. However, existing methods rely heavily …
Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models
Continual learning (CL) with Vision-Language Models (VLMs) has overcome the constraints
of traditional CL, which only focuses on previously encountered classes. During the CL of …
of traditional CL, which only focuses on previously encountered classes. During the CL of …
Trends and challenges of real-time learning in large language models: A critical review
M Jovanovic, P Voss - arxiv preprint arxiv:2404.18311, 2024 - arxiv.org
Real-time learning concerns the ability of learning systems to acquire knowledge over time,
enabling their adaptation and generalization to novel tasks. It is a critical ability for …
enabling their adaptation and generalization to novel tasks. It is a critical ability for …