Abstract Machine Learning (ML) is one of the most exciting and dynamic areas of modern
research and application. The purpose of this review is to provide an introduction to the core …

The recent striking success of deep neural networks in machine learning raises profound
questions about the theoretical principles underlying their success. For example, what can …

When and why can a neural network be successfully trained? This article provides an
overview of optimization algorithms and theory for training neural networks. First, we discuss …

This review addresses the method of explicit calculations of interatomic exchange
interactions of magnetic materials. This involves exchange mechanisms normally referred to …

Extreme value statistics (EVS) concerns the study of the statistics of the maximum or the
minimum of a set of random variables. This is an important problem for any time-series and …

Supervised deep learning involves the training of neural networks with a large number N of
parameters. For large enough N, in the so-called over-parametrized regime, one can …

The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often
considered to be Gaussian in the large data regime by assuming that the classical central …

Optimization is a critical component in deep learning. We think optimization for neural
networks is an interesting topic for theoretical research due to various reasons. First, its …

Random Matrix Theory (RMT) is applied to analyze the weight matrices of Deep Neural
Networks (DNNs), including both production quality, pre-trained models such as AlexNet …

Two distinct limits for deep learning have been derived as the network width h→∞,
depending on how the weights of the last layer scale with h. In the neural tangent Kernel …