A comprehensive survey of continual learning: theory, method and application
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …
Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality
Prompt-based continual learning is an emerging direction in leveraging pre-trained
knowledge for downstream continual learning, and has almost reached the performance …
knowledge for downstream continual learning, and has almost reached the performance …
Ranpac: Random projections and pre-trained models for continual learning
Continual learning (CL) aims to incrementally learn different tasks (such as classification) in
a non-stationary data stream without forgetting old ones. Most CL works focus on tackling …
a non-stationary data stream without forgetting old ones. Most CL works focus on tackling …
Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning
Exemplar-free class-incremental learning (CIL) poses several challenges since it prohibits
the rehearsal of data from previous tasks and thus suffers from catastrophic forgetting …
the rehearsal of data from previous tasks and thus suffers from catastrophic forgetting …
Continual learning: Applications and the road forward
Continual learning is a subfield of machine learning, which aims to allow machine learning
models to continuously learn on new data, by accumulating knowledge without forgetting …
models to continuously learn on new data, by accumulating knowledge without forgetting …
Online continual learning without the storage constraint
Traditional online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout an agent's …
catastrophic forgetting with fixed and limited storage allocation throughout an agent's …
Exemplar-free continual representation learning via learnable drift compensation
Exemplar-free class-incremental learning using a backbone trained from scratch and
starting from a small first task presents a significant challenge for continual representation …
starting from a small first task presents a significant challenge for continual representation …
Weighted ensemble models are strong continual learners
In this work, we study the problem of continual learning (CL) where the goal is to learn a
model on a sequence of tasks, under the assumption that the data from the previous tasks …
model on a sequence of tasks, under the assumption that the data from the previous tasks …
HIDE-PET: continual learning via hierarchical decomposition of parameter-efficient tuning
The deployment of pre-trained models (PTMs) has greatly advanced the field of continual
learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgetting …
learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgetting …
Continually learning representations at scale
Many widely used continual learning benchmarks follow a protocol that starts from an
untrained, randomly initialized model that needs to sequentially learn a number of incoming …
untrained, randomly initialized model that needs to sequentially learn a number of incoming …