Abstract Machine Learning (ML) is one of the most exciting and dynamic areas of modern
research and application. The purpose of this review is to provide an introduction to the core …

Substantial progress has been made recently on develo** provably accurate and efficient
algorithms for low-rank matrix factorization via nonconvex optimization. While conventional …

In deep learning, different kinds of deep networks typically need different optimizers, which
have to be chosen after multiple trials, making the training process inefficient. To relieve this …

Training large neural networks requires distributing learning across multiple workers, where
the cost of communicating gradients can be a significant bottleneck. signSGD alleviates this …

In this paper, we propose a new technique named\textit {Stochastic Path-Integrated
Differential EstimatoR}(SPIDER), which can be used to track many deterministic quantities of …

This paper studies a class of adaptive gradient based momentum algorithms that update the
search directions and learning rates simultaneously using past gradients. This class, which …

This work attempts to provide a plausible theoretical framework that aims to interpret modern
deep (convolutional) networks from the principles of data compression and discriminative …

Policy gradient (PG) methods have been one of the most essential ingredients of
reinforcement learning, with application in a variety of domains. In spite of the empirical …

This paper considers the problem of solving systems of quadratic equations, namely,
recovering an object of interest x^ ♮ ∈ R^ nx♮∈ R n from m quadratic equations/samples …

We study nonconvex optimization problems, where the objective function is either an
average of n nonconvex functions or the expectation of some stochastic function. We …