A comprehensive survey of continual learning: theory, method and application

L Wang, X Zhang, H Su, J Zhu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

A comprehensive survey of forgetting in deep learning beyond continual learning

Z Wang, E Yang, L Shen… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Forgetting refers to the loss or deterioration of previously acquired knowledge. While
existing surveys on forgetting have primarily focused on continual learning, forgetting is a …

A survey on negative transfer

W Zhang, L Deng, L Zhang, D Wu - IEEE/CAA Journal of …, 2022 - ieeexplore.ieee.org
Transfer learning (TL) utilizes data or knowledge from one or more source domains to
facilitate learning in a target domain. It is particularly useful when the target domain has very …

On the stability-plasticity dilemma of class-incremental learning

D Kim, B Han - Proceedings of the IEEE/CVF Conference …, 2023 - openaccess.thecvf.com
A primary goal of class-incremental learning is to strike a balance between stability and
plasticity, where models should be both stable enough to retain knowledge learned from …

Wide neural networks forget less catastrophically

SI Mirzadeh, A Chaudhry, D Yin, H Hu… - International …, 2022 - proceedings.mlr.press
A primary focus area in continual learning research is alleviating the" catastrophic forgetting"
problem in neural networks by designing new algorithms that are more robust to the …

Continual learning in the teacher-student setup: Impact of task similarity

S Lee, S Goldt, A Saxe - International Conference on …, 2021 - proceedings.mlr.press
Continual learning {—} the ability to learn many tasks in sequence {—} is critical for artificial
learning systems. Yet standard training methods for deep networks often suffer from …

Deep reinforcement and infomax learning

B Mazoure, R Tachet des Combes… - Advances in …, 2020 - proceedings.neurips.cc
We posit that a reinforcement learning (RL) agent will perform better when it uses
representations that are better at predicting the future, particularly in terms of few-shot …

Theory on forgetting and generalization of continual learning

S Lin, P Ju, Y Liang, N Shroff - International Conference on …, 2023 - proceedings.mlr.press
Continual learning (CL), which aims to learn a sequence of tasks, has attracted significant
recent attention. However, most work has focused on the experimental performance of CL …

The ideal continual learner: An agent that never forgets

L Peng, P Giampouras, R Vidal - … Conference on Machine …, 2023 - proceedings.mlr.press
The goal of continual learning is to find a model that solves multiple learning tasks which are
presented sequentially to the learner. A key challenge in this setting is that the learner may" …

How catastrophic can catastrophic forgetting be in linear regression?

I Evron, E Moroshko, R Ward… - … on Learning Theory, 2022 - proceedings.mlr.press
To better understand catastrophic forgetting, we study fitting an overparameterized linear
model to a sequence of tasks with different input distributions. We analyze how much the …