Stochastic optimization lies at the heart of machine learning, and its cornerstone is
stochastic gradient descent (SGD), a method introduced over 60 years ago. The last eight …

We study nonconvex optimization problems, where the objective function is either an
average of n nonconvex functions or the expectation of some stochastic function. We …

We present a unified framework to analyze the global convergence of Langevin dynamics
based algorithms for nonconvex finite-sum optimization with $ n $ component functions. At …

Motivated by recent increased interest in optimization algorithms for non-convex
optimization in application to training deep neural networks and other optimization problems …

We consider the problem of minimizing a continuous function given given access to a
natural quantum generalization of a stochastic gradient oracle. We provide two new …

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of
functions satisfying gradient dominance property with $1\le\alpha\le2 $ which holds in a …

Scalable and efficient distributed learning is one of the main driving forces behind the recent
rapid advancement of machine learning and artificial intelligence. One prominent feature of …

We study the smooth minimax optimization problem $\min_ {\bf x}\max_ {\bf y} f ({\bf x},{\bf y})
$, where $ f $ is $\ell $-smooth, strongly-concave in ${\bf y} $ but possibly nonconvex in ${\bf …

The right to be forgotten has been legislated in many countries, but its enforcement in the AI
industry would cause unbearable costs. When single data deletion requests come …

Adaptive regularization with cubics (ARC) is an algorithm for unconstrained, non-convex
optimization. Akin to the trust-region method, its iterations can be thought of as approximate …