Learning curves of generic features maps for realistic datasets with a teacher-student model
Teacher-student models provide a framework in which the typical-case performance of high-
dimensional supervised learning can be described in closed form. The assumptions of …
dimensional supervised learning can be described in closed form. The assumptions of …
Generalisation error in learning with random features and the hidden manifold model
We study generalised linear regression and classification for a synthetically generated
dataset encompassing different problems of interest, such as learning with random features …
dataset encompassing different problems of interest, such as learning with random features …
[BOOK][B] Statistical mechanics of learning
A Engel - 2001 - books.google.com
Learning is one of the things that humans do naturally, and it has always been a challenge
for us to understand the process. Nowadays this challenge has another dimension as we try …
for us to understand the process. Nowadays this challenge has another dimension as we try …
Double trouble in double descent: Bias and variance (s) in the lazy regime
Deep neural networks can achieve remarkable generalization performances while
interpolating the training data. Rather than the U-curve emblematic of the bias-variance …
interpolating the training data. Rather than the U-curve emblematic of the bias-variance …
A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors
T Tanaka - IEEE Transactions on Information theory, 2002 - ieeexplore.ieee.org
We present a theory, based on statistical mechanics, to evaluate analytically the
performance of uncoded, fully synchronous, randomly spread code-division multiple-access …
performance of uncoded, fully synchronous, randomly spread code-division multiple-access …
Are Gaussian data all you need? The extents and limits of universality in high-dimensional generalized linear estimation
In this manuscript we consider the problem of generalized linear estimation on Gaussian
mixture data with labels given by a single-index model. Our first result is a sharp asymptotic …
mixture data with labels given by a single-index model. Our first result is a sharp asymptotic …
Generalization error rates in kernel regression: The crossover from the noiseless to noisy regime
In this manuscript we consider Kernel Ridge Regression (KRR) under the Gaussian design.
Exponents for the decay of the excess generalization error of KRR have been reported in …
Exponents for the decay of the excess generalization error of KRR have been reported in …
Triple descent and the two kinds of overfitting: Where & why do they appear?
A recent line of research has highlighted the existence of a``double descent''phenomenon in
deep learning, whereby increasing the number of training examples N causes the …
deep learning, whereby increasing the number of training examples N causes the …
A theory of non-linear feature learning with one gradient step in two-layer neural networks
Feature learning is thought to be one of the fundamental reasons for the success of deep
neural networks. It is rigorously known that in two-layer fully-connected neural networks …
neural networks. It is rigorously known that in two-layer fully-connected neural networks …
Estimating dataset size requirements for classifying DNA microarray data
A statistical methodology for estimating dataset size requirements for classifying microarray
data using learning curves is introduced. The goal is to use existing classification results to …
data using learning curves is introduced. The goal is to use existing classification results to …