High-dimensional learning of narrow neural networks
H Cui - arxiv preprint arxiv:2409.13904, 2024 - arxiv.org
Recent years have been marked with the fast-pace diversification and increasing ubiquity of
machine learning applications. Yet, a firm theoretical understanding of the surprising …
machine learning applications. Yet, a firm theoretical understanding of the surprising …
Scaling and renormalization in high-dimensional regression
This paper presents a succinct derivation of the training and generalization performance of a
variety of high-dimensional ridge regression models using the basic tools of random matrix …
variety of high-dimensional ridge regression models using the basic tools of random matrix …
On double-descent in uncertainty quantification in overparametrized models
Uncertainty quantification is a central challenge in reliable and trustworthy machine
learning. Naive measures such as last-layer scores are well-known to yield overconfident …
learning. Naive measures such as last-layer scores are well-known to yield overconfident …
Asymptotics of feature learning in two-layer networks after one gradient-step
In this manuscript we investigate the problem of how two-layer neural networks learn
features from data, and improve over the kernel regime, after being trained with a single …
features from data, and improve over the kernel regime, after being trained with a single …
Bayes-optimal learning of an extensive-width neural network from quadratically many samples
We consider the problem of learning a target function corresponding to a single hidden layer
neural network, with a quadratic activation function after the first layer, and random weights …
neural network, with a quadratic activation function after the first layer, and random weights …
Classification of heavy-tailed features in high dimensions: a superstatistical approach
We characterise the learning of a mixture of two clouds of data points with generic centroids
via empirical risk minimisation in the high dimensional regime, under the assumptions of …
via empirical risk minimisation in the high dimensional regime, under the assumptions of …
Multinomial logistic regression: Asymptotic normality on null covariates in high-dimensions
This paper investigates the asymptotic distribution of the maximum-likelihood estimate
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …
(MLE) in multinomial logistic models in the high-dimensional regime where dimension and …
Universality of max-margin classifiers
Maximum margin binary classification is one of the most fundamental algorithms in machine
learning, yet the role of featurization maps and the high-dimensional asymptotics of the …
learning, yet the role of featurization maps and the high-dimensional asymptotics of the …
Gaussian universality of perceptrons with random labels
While classical in many theoretical settings—and in particular in statistical physics-inspired
works—the assumption of Gaussian iid input data is often perceived as a strong limitation in …
works—the assumption of Gaussian iid input data is often perceived as a strong limitation in …
Fitting an ellipsoid to random points: predictions using the replica method
We consider the problem of fitting a centered ellipsoid to n standard Gaussian random
vectors in, as with. It has been conjectured that this problem is, with high probability …
vectors in, as with. It has been conjectured that this problem is, with high probability …