We identify and formalize a fundamental gradient descent phenomenon resulting in a
learning proclivity in over-parameterized neural networks. Gradient Starvation arises when …

Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics
of neural networks as they train on single classification tasks. Our goal is to understand …

Spurious correlations pose a major challenge for robust machine learning. Models trained
with empirical risk minimization (ERM) may learn to rely on correlations between class …

Our understanding of the generalization capabilities of neural networks NNs is still
incomplete. Prevailing explanations are based on implicit biases of gradient descent GD but …

State representation learning, or the ability to capture latent generative factors of an
environment is crucial for building intelligent agents that can perform a wide variety of tasks …

Deep-learning models can extract a rich assortment of features from data. Which features a
model uses depends not only on\emph {predictivity}--how reliably a feature indicates …

Research has shown that neural models implicitly encode linguistic features, but there has
been no research showing\emph {how} these encodings arise as the models are trained …

Recent studies suggest that deep learning models' inductive bias towards favoring simpler
features may be an origin of shortcut learning. Yet, there has been limited focus on …

A leading hypothesis for the surprising generalization of neural networks is that the
dynamics of gradient descent bias the model towards simple solutions, by searching through …

A critical problem in deep learning is that systems learn inappropriate biases, resulting in
their inability to perform well on minority groups. This has led to the creation of multiple …