Stochastic optimization lies at the heart of machine learning, and its cornerstone is
stochastic gradient descent (SGD), a method introduced over 60 years ago. The last eight …

We introduce a new and increasingly relevant setting for distributed optimization in machine
learning, where the data defining the optimization are unevenly distributed over an …

We propose a stochastic variant of the classical Polyak step-size (Polyak, 1987) commonly
used in the subgradient method. Although computing the Polyak step-size requires …

We develop a class of algorithms, as variants of the stochastically controlled stochastic
gradient (SCSG) methods, for the smooth nonconvex finite-sum optimization problem. Only …

Deep neural networks are commonly trained using stochastic non-convex optimization
procedures, which are driven by gradient information estimated on fractions (batches) of the …

An up-to-date account of the interplay between optimization and machine learning,
accessible to students and researchers in both communities. The interplay between …

We study nonconvex optimization problems, where the objective function is either an
average of n nonconvex functions or the expectation of some stochastic function. We …

In this paper, we propose a novel reinforcement-learning algorithm consisting in a stochastic
variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs) …

One of the major issues in stochastic gradient descent (SGD) methods is how to choose an
appropriate step size while running the algorithm. Since the traditional line search technique …

We consider convex-concave saddle-point problems where the objective functions may be
split in many components, and extend recent stochastic variance reduction methods (such …