We consider a family of algorithms that successively sample and minimize simple stochastic
models of the objective function. We show that under reasonable conditions on …

Min–max problems have broad applications in machine learning, including learning with
non-decomposable loss and learning with robustness to data distribution. Convex–concave …

We consider minimization of stochastic functionals that are compositions of a (potentially)
nonsmooth convex function h and smooth function c and, more generally, stochastic weakly …

Conditional stochastic optimization covers a variety of applications ranging from invariant
learning and causal inference to meta-learning. However, constructing unbiased gradient …

Standard stochastic optimization methods are brittle, sensitive to stepsize choice and other
algorithmic parameters, and they exhibit instability outside of well-behaved families of …

Subgradient methods converge linearly on a convex function that grows sharply away from
its solution set. In this work, we show that the same is true for sharp functions that are only …

We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which
the objective function is weakly convex in the ambient Euclidean space. Such problems are …

We prove that the proximal stochastic subgradient method, applied to a weakly convex
problem, drives the gradient of the Moreau envelope to zero at the rate $ O (k^{-1/4}) $. As a …

Stochastic gradient methods with momentum are widely used in applications and at the core
of optimization subroutines in many popular machine learning libraries. However, their …

We develop a new primitive for stochastic optimization: a low-bias, low-cost estimator of the
minimizer $ x_\star $ of any Lipschitz strongly-convex function $ f $. In particular, we use a …