Continual learning of large language models: A comprehensive survey

H Shi, Z Xu, H Wang, W Qin, W Wang, Y Wang… - arxiv preprint arxiv …, 2024 - arxiv.org
The recent success of large language models (LLMs) trained on static, pre-collected,
general datasets has sparked numerous research directions and applications. One such …

A comprehensive survey of forgetting in deep learning beyond continual learning

Z Wang, E Yang, L Shen… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Forgetting refers to the loss or deterioration of previously acquired knowledge. While
existing surveys on forgetting have primarily focused on continual learning, forgetting is a …

Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need

DW Zhou, ZW Cai, HJ Ye, DC Zhan, Z Liu - arxiv preprint arxiv …, 2023 - arxiv.org
Class-incremental learning (CIL) aims to adapt to emerging new classes without forgetting
old ones. Traditional CIL models are trained from scratch to continually acquire knowledge …

Class-incremental learning: A survey

DW Zhou, QW Wang, ZH Qi, HJ Ye… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
Deep models, eg, CNNs and Vision Transformers, have achieved impressive achievements
in many vision tasks in the closed world. However, novel classes emerge from time to time in …

Continual learning: Applications and the road forward

E Verwimp, R Aljundi, S Ben-David, M Bethge… - arxiv preprint arxiv …, 2023 - arxiv.org
Continual learning is a subfield of machine learning, which aims to allow machine learning
models to continuously learn on new data, by accumulating knowledge without forgetting …

Diffclass: Diffusion-based class incremental learning

Z Meng, J Zhang, C Yang, Z Zhan, P Zhao… - European Conference on …, 2024 - Springer
Abstract Class Incremental Learning (CIL) is challenging due to catastrophic forgetting. On
top of that, exemplar-free CIL is even more challenging due to forbidden access to data of …

Learning without forgetting for vision-language models

DW Zhou, Y Zhang, J Ning, HJ Ye, DC Zhan… - arxiv preprint arxiv …, 2023 - arxiv.org
Class-Incremental Learning (CIL) or continual learning is a desired capability in the real
world, which requires a learning system to adapt to new tasks without forgetting former ones …

Pilot: A pre-trained model-based continual learning toolbox

HL Sun, DW Zhou, HJ Ye, DC Zhan - arxiv preprint arxiv:2309.07117, 2023 - arxiv.org
While traditional machine learning can effectively tackle a wide range of problems, it
primarily operates within a closed-world setting, which presents limitations when dealing …

A collective AI via lifelong learning and sharing at the edge

A Soltoggio, E Ben-Iwhiwhu, V Braverman… - Nature Machine …, 2024 - nature.com
One vision of a future artificial intelligence (AI) is where many separate units can learn
independently over a lifetime and share their knowledge with each other. The synergy …

Cost-effective on-device continual learning over memory hierarchy with Miro

X Ma, S Jeong, M Zhang, D Wang, J Choi… - Proceedings of the 29th …, 2023 - dl.acm.org
Continual learning (CL) trains NN models incrementally from a continuous stream of tasks.
To remember previously learned knowledge, prior studies store old samples over a memory …