Following the remarkable success of the AlphaGO series, 2019 was a booming year that
witnessed significant advances in multi-agent reinforcement learning (MARL) techniques …

Transformer architecture has shown impressive performance in multiple research domains
and has become the backbone of many neural network models. However, there is limited …

We describe the new field of the mathematical analysis of deep learning. This field emerged
around a list of research questions that were not answered within the classical framework of …

We propose Joint MLP/Attention (JoMA) dynamics, a novel mathematical framework to
understand the training procedure of multilayer Transformer architectures. This is achieved …

Abstract Mean-Field Game (MFG) serves as a crucial mathematical framework in modeling
the collective behavior of individual agents interacting stochastically with a large population …

Neural Architecture Search (NAS) is widely used to automatically obtain the neural network
with the best performance among a large number of candidate architectures. To reduce the …

A three-hidden-layer neural network with super approximation power is introduced. This
network is built with the floor function (⌊ x⌋), the exponential function (2 x), the step function …

We develop a mathematically rigorous framework for multilayer neural networks in the mean
field regime. As the network's widths increase, the network's learning trajectory is shown to …

The problem of solving partial differential equations (PDEs) can be formulated into a least-
squares minimization problem, where neural networks are used to parametrize PDE …

We revisit the canonical problem of learning a single neuron with ReLU activation under
Gaussian input with square loss. We particularly focus on the over-parameterization setting …