A collective AI via lifelong learning and sharing at the edge

A Soltoggio, E Ben-Iwhiwhu, V Braverman… - Nature Machine …, 2024 - nature.com
One vision of a future artificial intelligence (AI) is where many separate units can learn
independently over a lifetime and share their knowledge with each other. The synergy …

Continual learning: Applications and the road forward

E Verwimp, R Aljundi, S Ben-David, M Bethge… - arxiv preprint arxiv …, 2023 - arxiv.org
Continual learning is a subfield of machine learning, which aims to allow machine learning
models to continuously learn on new data, by accumulating knowledge without forgetting …

Trends and challenges of real-time learning in large language models: A critical review

M Jovanovic, P Voss - arxiv preprint arxiv:2404.18311, 2024 - arxiv.org
Real-time learning concerns the ability of learning systems to acquire knowledge over time,
enabling their adaptation and generalization to novel tasks. It is a critical ability for …

Simple and scalable strategies to continually pre-train large language models

A Ibrahim, B Thérien, K Gupta, ML Richter… - arxiv preprint arxiv …, 2024 - arxiv.org
Large language models (LLMs) are routinely pre-trained on billions of tokens, only to start
the process over again once new data becomes available. A much more efficient solution is …

A Practitioner's Guide to Continual Multimodal Pretraining

K Roth, V Udandarao, S Dziadzio, A Prabhu… - arxiv preprint arxiv …, 2024 - arxiv.org
Multimodal foundation models serve numerous applications at the intersection of vision and
language. Still, despite being pretrained on extensive data, they become outdated over time …

Reserving embedding space for new fault types: A new continual learning method for bearing fault diagnosis

H Zhu, C Shen, L Li, D Wang, W Huang… - Reliability Engineering & …, 2024 - Elsevier
In complex operating environments, rotating equipment may continually generate new fault
categories, affecting the safety of equipment operation, and the number of collected fault …

SIESTA: Efficient online continual learning with sleep

MY Harun, J Gallardo, TL Hayes, R Kemker… - arxiv preprint arxiv …, 2023 - arxiv.org
In supervised continual learning, a deep neural network (DNN) is updated with an ever-
growing data stream. Unlike the offline setting where data is shuffled, we cannot make any …

Grasp: A rehearsal policy for efficient online continual learning

MY Harun, J Gallardo, J Chen, C Kanan - arxiv preprint arxiv:2308.13646, 2023 - arxiv.org
Continual learning (CL) in deep neural networks (DNNs) involves incrementally
accumulating knowledge in a DNN from a growing data stream. A major challenge in CL is …

Resource-efficient continual learning for personalized online seizure detection

A Shahbazinia, F Ponzina, JA Miranda… - 2024 46th Annual …, 2024 - ieeexplore.ieee.org
Epilepsy, a major neurological disease, requires careful diagnosis and treatment. However,
the detection of epileptic seizures remains a significant challenge. Current clinical practice …

Overcoming the stability gap in continual learning

MY Harun, C Kanan - arxiv preprint arxiv:2306.01904, 2023 - arxiv.org
Pre-trained deep neural networks (DNNs) are being widely deployed by industry for making
business decisions and to serve users; however, a major problem is model decay, where the …