One vision of a future artificial intelligence (AI) is where many separate units can learn
independently over a lifetime and share their knowledge with each other. The synergy …

Lifelong learning—an agent's ability to learn throughout its lifetime—is a hallmark of
biological learning systems and a central challenge for artificial intelligence (AI). The …

Large language models (LLMs) are routinely pre-trained on billions of tokens, only to start
the process over again once new data becomes available. A much more efficient solution is …

Large language models (LLMs) are routinely pre-trained on billions of tokens, only to restart
the process over again once new data becomes available. A much cheaper and more …

Recent studies have demonstrated the potency of leveraging prompts in Transformers for
continual learning (CL). Nevertheless, employing a discrete key-prompt bottleneck can lead …

This book chapter delves into the dynamics of continual learning, which is the process of
incrementally learning from a non-stationary stream of data. Although continual learning is a …

Online continual learning aims to get closer to a live learning experience by learning directly
on a stream of data with temporally shifting distribution and by storing a minimum amount of …

Traditional online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout an agent's …

Deep representation learning methods struggle with continual learning, suffering from both
catastrophic forgetting of useful units and loss of plasticity, often due to rigid and unuseful …

Continual learning (CL) aims to help deep neural networks learn new knowledge while
retaining what has been learned. Owing to their powerful generalizability, pre-trained vision …