A comprehensive survey of continual learning: theory, method and application

L Wang, X Zhang, H Su, J Zhu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality

L Wang, J **e, X Zhang, M Huang… - Advances in Neural …, 2024 - proceedings.neurips.cc
Prompt-based continual learning is an emerging direction in leveraging pre-trained
knowledge for downstream continual learning, and has almost reached the performance …

Ranpac: Random projections and pre-trained models for continual learning

MD McDonnell, D Gong, A Parvaneh… - Advances in …, 2024 - proceedings.neurips.cc
Continual learning (CL) aims to incrementally learn different tasks (such as classification) in
a non-stationary data stream without forgetting old ones. Most CL works focus on tackling …

Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning

D Goswami, Y Liu, B Twardowski… - Advances in Neural …, 2024 - proceedings.neurips.cc
Exemplar-free class-incremental learning (CIL) poses several challenges since it prohibits
the rehearsal of data from previous tasks and thus suffers from catastrophic forgetting …

Continual learning: Applications and the road forward

E Verwimp, R Aljundi, S Ben-David, M Bethge… - arxiv preprint arxiv …, 2023 - arxiv.org
Continual learning is a subfield of machine learning, which aims to allow machine learning
models to continuously learn on new data, by accumulating knowledge without forgetting …

Online continual learning without the storage constraint

A Prabhu, Z Cai, P Dokania, P Torr, V Koltun… - arxiv preprint arxiv …, 2023 - arxiv.org
Traditional online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout an agent's …

Exemplar-free continual representation learning via learnable drift compensation

A Gomez-Villa, D Goswami, K Wang… - … on Computer Vision, 2024 - Springer
Exemplar-free class-incremental learning using a backbone trained from scratch and
starting from a small first task presents a significant challenge for continual representation …

Weighted ensemble models are strong continual learners

IE Marouf, S Roy, E Tartaglione… - European Conference on …, 2024 - Springer
In this work, we study the problem of continual learning (CL) where the goal is to learn a
model on a sequence of tasks, under the assumption that the data from the previous tasks …

HIDE-PET: continual learning via hierarchical decomposition of parameter-efficient tuning

L Wang, J **e, X Zhang, H Su, J Zhu - arxiv preprint arxiv:2407.05229, 2024 - arxiv.org
The deployment of pre-trained models (PTMs) has greatly advanced the field of continual
learning (CL), enabling positive knowledge transfer and resilience to catastrophic forgetting …

Continually learning representations at scale

A Galashov, J Mitrovic, D Tirumala… - Conference on …, 2023 - proceedings.mlr.press
Many widely used continual learning benchmarks follow a protocol that starts from an
untrained, randomly initialized model that needs to sequentially learn a number of incoming …