High-dimensional learning of narrow neural networks

H Cui - arxiv preprint arxiv:2409.13904, 2024 - arxiv.org
Recent years have been marked with the fast-pace diversification and increasing ubiquity of
machine learning applications. Yet, a firm theoretical understanding of the surprising …

Scaling and renormalization in high-dimensional regression

A Atanasov, JA Zavatone-Veth, C Pehlevan - arxiv preprint arxiv …, 2024 - arxiv.org
This paper presents a succinct derivation of the training and generalization performance of a
variety of high-dimensional ridge regression models using the basic tools of random matrix …

On double-descent in uncertainty quantification in overparametrized models

L Clarté, B Loureiro, F Krzakala… - International …, 2023 - proceedings.mlr.press
Uncertainty quantification is a central challenge in reliable and trustworthy machine
learning. Naive measures such as last-layer scores are well-known to yield overconfident …

Asymptotics of feature learning in two-layer networks after one gradient-step

H Cui, L Pesce, Y Dandi, F Krzakala, YM Lu… - arxiv preprint arxiv …, 2024 - arxiv.org
In this manuscript we investigate the problem of how two-layer neural networks learn
features from data, and improve over the kernel regime, after being trained with a single …

Bayes-optimal learning of an extensive-width neural network from quadratically many samples

A Maillard, E Troiani, S Martin, F Krzakala… - arxiv preprint arxiv …, 2024 - arxiv.org
We consider the problem of learning a target function corresponding to a single hidden layer
neural network, with a quadratic activation function after the first layer, and random weights …

Classification of heavy-tailed features in high dimensions: a superstatistical approach

U Adomaityte, G Sicuro, P Vivo - Advances in Neural …, 2023 - proceedings.neurips.cc
We characterise the learning of a mixture of two clouds of data points with generic centroids
via empirical risk minimisation in the high dimensional regime, under the assumptions of …

Multinomial logistic regression: Asymptotic normality on null covariates in high-dimensions

K Tan, PC Bellec - Advances in Neural Information …, 2024 - proceedings.neurips.cc
This paper investigates the asymptotic distribution of the maximum-likelihood estimate
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …

Universality of max-margin classifiers

A Montanari, F Ruan, B Saeed, Y Sohn - arxiv preprint arxiv:2310.00176, 2023 - arxiv.org
Maximum margin binary classification is one of the most fundamental algorithms in machine
learning, yet the role of featurization maps and the high-dimensional asymptotics of the …

Gaussian universality of perceptrons with random labels

F Gerace, F Krzakala, B Loureiro, L Stephan… - Physical Review E, 2024 - APS
While classical in many theoretical settings—and in particular in statistical physics-inspired
works—the assumption of Gaussian iid input data is often perceived as a strong limitation in …

Fitting an ellipsoid to random points: predictions using the replica method

A Maillard, D Kunisky - IEEE Transactions on Information …, 2024 - ieeexplore.ieee.org
We consider the problem of fitting a centered ellipsoid to n standard Gaussian random
vectors in, as with. It has been conjectured that this problem is, with high probability …