In machine learning applications, it is well known that carefully designed learning rate (step
size) schedules can significantly improve the convergence of commonly used first-order …

The {\L} ojasiewicz inequality shows that H\" olderian error bounds on the minimum of
convex optimization problems hold almost generically. Here, we clarify results of\citet …

First-order primal-dual methods are appealing for their low memory overhead, fast iterations,
and effective parallelization. However, they are often slow at finding high accuracy solutions …

In this paper, we study the efficiency of a Restarted SubGradient (RSG) method that
periodically restarts the standard subgradient method (SG). We show that, when applied to a …

This paper settles an open and challenging question pertaining to the design of simple and
optimal high-order methods for solving smooth and monotone variational inequalities (VIs) …

Stochastic gradient descent (SGD) algorithms, with constant momentum and its variants
such as Adam, are the optimization methods of choice for training deep neural networks …

Stochastic (sub) gradient methods require step size schedule tuning to perform well in
practice. Classical tuning strategies decay the step size polynomially and lead to optimal …

Convex quadratic programming (QP) is an important class of optimization problem with wide
applications in practice. The classic QP solvers are based on either simplex or barrier …

The purpose of this manuscript is to derive new convergence results for several subgradient
methods applied to minimizing nonsmooth convex functions with Hölderian growth. The …

A subgradient method is presented for solving general convex optimization problems, the
main requirement being that a strictly feasible point is known. A feasible sequence of iterates …