To cope with real-world dynamics, an intelligent system needs to incrementally acquire,
update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as …

The recent success of large language models (LLMs) trained on static, pre-collected,
general datasets has sparked numerous research directions and applications. One such …

Despite their wide adoption, the underlying training and memorization dynamics of very
large language models is not well understood. We empirically study exact memorization in …

Large language models (LLMs) are routinely pre-trained on billions of tokens, only to start
the process over again once new data becomes available. A much more efficient solution is …

A large body of research in continual learning is devoted to overcoming the catastrophic
forgetting of neural networks by designing new algorithms that are robust to the distribution …

The goal of continual learning is to find a model that solves multiple learning tasks which are
presented sequentially to the learner. A key challenge in this setting is that the learner may" …

As the number of large language models (LLMs) released to the public grows, there is a
pressing need to understand the safety implications associated with these models learning …

To better understand catastrophic forgetting, we study fitting an overparameterized linear
model to a sequence of tasks with different input distributions. We analyze how much the …

We study the membership inference (MI) attack against classifiers, where the attacker's goal
is to determine whether a data instance was used for training the classifier. Through …

Continual learning requires incremental compatibility with a sequence of tasks. However,
the design of model architecture remains an open question: In general, learning all tasks …