AI alignment aims to make AI systems behave in line with human intentions and values. As
AI systems grow more capable, so do risks from misalignment. To provide a comprehensive …

Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

In this paper we look into the conjecture of Entezari et al.(2021) which states that if the
permutation invariance of neural networks is taken into account, then there is likely no loss …

We study neural network loss landscapes through the lens of mode connectivity, the
observation that minimizers of neural networks retrieved via training on a dataset are …

Distillation strategies are currently the primary approaches for mitigating forgetting in class
incremental learning (CIL). Existing methods generally inherit previous knowledge from a …

The energy landscape of high-dimensional non-convex optimization problems is crucial to
understanding the effectiveness of modern deep neural network architectures. Recent works …

Many algorithms and observed phenomena in deep learning appear to be affected by
parameter symmetries--transformations of neural network parameters that do not change the …

Studying the interplay between the geometry of the loss landscape and the optimization
trajectories of simple neural networks is a fundamental step for understanding their behavior …

Current deep neural networks are highly overparameterized (up to billions of connection
weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient …

Empirical studies of the loss landscape of deep networks have revealed that many local
minima are connected through low-loss valleys. Yet, little is known about the theoretical …