Learning curves of generic features maps for realistic datasets with a teacher-student model

B Loureiro, C Gerbelot, H Cui, S Goldt… - Advances in …, 2021 - proceedings.neurips.cc
Teacher-student models provide a framework in which the typical-case performance of high-
dimensional supervised learning can be described in closed form. The assumptions of …

[KNIHA][B] Random matrix methods for machine learning

R Couillet, Z Liao - 2022 - books.google.com
This book presents a unified theory of random matrices for applications in machine learning,
offering a large-dimensional data vision that exploits concentration and universality …

Geometric dataset distances via optimal transport

D Alvarez-Melis, N Fusi - Advances in Neural Information …, 2020 - proceedings.neurips.cc
The notion of task similarity is at the core of various machine learning paradigms, such as
domain adaptation and meta-learning. Current methods to quantify it are often heuristic …

Generalisation error in learning with random features and the hidden manifold model

F Gerace, B Loureiro, F Krzakala… - International …, 2020 - proceedings.mlr.press
We study generalised linear regression and classification for a synthetically generated
dataset encompassing different problems of interest, such as learning with random features …

Universality laws for high-dimensional learning with random features

H Hu, YM Lu - IEEE Transactions on Information Theory, 2022 - ieeexplore.ieee.org
We prove a universality theorem for learning with random features. Our result shows that, in
terms of training and generalization errors, a random feature model with a nonlinear …

Modeling the influence of data structure on learning in neural networks: The hidden manifold model

S Goldt, M Mézard, F Krzakala, L Zdeborová - Physical Review X, 2020 - APS
Understanding the reasons for the success of deep neural networks trained using stochastic
gradient-based methods is a key open problem for the nascent theory of deep learning. The …

The gaussian equivalence of generative models for learning with shallow neural networks

S Goldt, B Loureiro, G Reeves… - Mathematical and …, 2022 - proceedings.mlr.press
Understanding the impact of data structure on the computational tractability of learning is a
key challenge for the theory of neural networks. Many theoretical works do not explicitly …

A random matrix analysis of random fourier features: beyond the gaussian kernel, a precise phase transition, and the corresponding double descent

Z Liao, R Couillet, MW Mahoney - Advances in Neural …, 2020 - proceedings.neurips.cc
This article characterizes the exact asymptotics of random Fourier feature (RFF) regression,
in the realistic setting where the number of data samples $ n $, their dimension $ p $, and …

[PDF][PDF] Learning gaussian mixtures with generalized linear models: Precise asymptotics in high-dimensions

B Loureiro, G Sicuro, C Gerbelot… - Advances in …, 2021 - proceedings.neurips.cc
Generalised linear models for multi-class classification problems are one of the fundamental
building blocks of modern machine learning tasks. In this manuscript, we characterise the …

Universality laws for gaussian mixtures in generalized linear models

Y Dandi, L Stephan, F Krzakala… - Advances in …, 2023 - proceedings.neurips.cc
A recent line of work in high-dimensional statistics working under the Gaussian mixture
hypothesis has led to a number of results in the context of empirical risk minimization …