Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review
The paper reviews and extends an emerging body of theoretical results on deep learning
including the conditions under which it can be exponentially better than shallow learning. A …
including the conditions under which it can be exponentially better than shallow learning. A …
Understanding deep learning (still) requires rethinking generalization
Despite their massive size, successful deep artificial neural networks can exhibit a
remarkably small gap between training and test performance. Conventional wisdom …
remarkably small gap between training and test performance. Conventional wisdom …
Error bounds for approximations with deep ReLU networks
D Yarotsky - Neural networks, 2017 - Elsevier
We study expressive power of shallow and deep neural networks with piece-wise linear
activation functions. We establish new rigorous upper and lower bounds for the network …
activation functions. We establish new rigorous upper and lower bounds for the network …
Nonparametric regression using deep neural networks with ReLU activation function
J Schmidt-Hieber - 2020 - projecteuclid.org
Nonparametric regression using deep neural networks with ReLU activation function Page 1
The Annals of Statistics 2020, Vol. 48, No. 4, 1875–1897 https://doi.org/10.1214/19-AOS1875 …
The Annals of Statistics 2020, Vol. 48, No. 4, 1875–1897 https://doi.org/10.1214/19-AOS1875 …
[BOOK][B] The nature of statistical learning theory
V Vapnik - 2013 - books.google.com
The aim of this book is to discuss the fundamental ideas which lie behind the statistical
theory of learning and generalization. It considers learning as a general problem of function …
theory of learning and generalization. It considers learning as a general problem of function …
[HTML][HTML] Universality of deep convolutional neural networks
DX Zhou - Applied and computational harmonic analysis, 2020 - Elsevier
Deep learning has been widely applied and brought breakthroughs in speech recognition,
computer vision, and many other domains. Deep neural network architectures and …
computer vision, and many other domains. Deep neural network architectures and …
[BOOK][B] Neural network learning: Theoretical foundations
M Anthony, PL Bartlett - 2009 - dl.acm.org
This important work describes recent theoretical advances in the study of artificial neural
networks. It explores probabilistic models of supervised learning problems, and addresses …
networks. It explores probabilistic models of supervised learning problems, and addresses …
Approximation theory of the MLP model in neural networks
A Pinkus - Acta numerica, 1999 - cambridge.org
In this survey we discuss various approximation-theoretic problems that arise in the
multilayer feedforward perceptron (MLP) model in neural networks. The MLP model is one of …
multilayer feedforward perceptron (MLP) model in neural networks. The MLP model is one of …
Regularization theory and neural networks architectures
We had previously shown that regularization principles lead to approximation schemes that
are equivalent to networks with one layer of hidden units, called regularization networks. In …
are equivalent to networks with one layer of hidden units, called regularization networks. In …
Graph neural networks exponentially lose expressive power for node classification
Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing
graph-structured data. However, it is known that they do not improve (or sometimes worsen) …
graph-structured data. However, it is known that they do not improve (or sometimes worsen) …