This monograph covers some recent advances in a range of acceleration techniques
frequently used in convex optimization. We first use quadratic optimization problems to …

We propose a new framework, inspired by random matrix theory, for analyzing the dynamics
of stochastic gradient descent (SGD) when both number of samples and dimensions are …

Average-case analysis computes the complexity of an algorithm averaged over all possible
inputs. Compared to worst-case analysis, it is more representative of the typical behavior of …

We develop a framework for the average-case analysis of random quadratic problems and
derive algorithms that are optimal under this analysis. This yields a new class of methods …

Polyak momentum (PM), also known as the heavy-ball method, is a widely used optimization
method that enjoys an asymptotic optimal worst-case complexity on quadratic objectives …

In distributed second order optimization, a standard strategy is to average many local
estimates, each of which is based on a small sketch or batch of the data. However, the local …

We propose a new randomized algorithm for solving L2-regularized least-squares problems
based on sketching. We consider two of the most popular random embeddings, namely …

This paper develops conformal inference methods to construct a confidence interval for the
frequency of a queried object in a very large discrete data set, based on a sketch with a …

Neural networks (NNs) have been extremely successful across many tasks in machine
learning. Quantization of NN weights has become an important topic due to its impact on …

We investigate iterative methods with randomized preconditioners for solving
overdetermined least-squares problems, where the preconditioners are based on a random …