We analyze feature learning in infinite-width neural networks trained with gradient flow
through a self-consistent dynamical field theory. We construct a collection of deterministic …

We analyze the dynamics of finite width effects in wide but finite feature learning neural
networks. Starting from a dynamical mean field theory description of infinite width deep …

Deep neural networks are widely known for their remarkable effectiveness across various
tasks, with the consensus that deeper networks implicitly learn more complex data …

We show that taking the width and depth to infinity in a deep neural network with skip
connections, when branches are scaled by $1/\sqrt {depth} $, result in the same covariance …

It is unclear how changing the learning rule of a deep neural network alters its learning
dynamics and representations. To gain insight into the relationship between learned …

We introduce a new analytical framework to quantify the changes in a machine learning
algorithm's output distribution following the inclusion of a few data points in its training set, a …

This work bridges two important concepts: the Neural Tangent Kernel (NTK), which captures
the evolution of deep neural networks (DNNs) during training, and the Neural Collapse (NC) …

In this paper, we study the infinite-depth limit of finite-width residual neural networks with
random Gaussian weights. With proper scaling, we show that by fixing the width and taking …

Feature-learning deep nets progressively collapse data to a regular low-dimensional
geometry. How this phenomenon emerges from collective action of nonlinearity, noise …

We analyze feature learning in infinite-width neural networks trained with gradient flow
through a self-consistent dynamical field theory. We construct a collection of deterministic …