Acceleration by stepsize hedging: Multi-step descent and the silver stepsize schedule
J Altschuler, P Parrilo - Journal of the ACM, 2023 - dl.acm.org
Can we accelerate the convergence of gradient descent without changing the algorithm—
just by judiciously choosing stepsizes? Surprisingly, we show that the answer is yes. Our …
just by judiciously choosing stepsizes? Surprisingly, we show that the answer is yes. Our …
Accelerated primal-dual gradient method for smooth and convex-concave saddle-point problems with bilinear coupling
In this paper we study the convex-concave saddle-point problem $\min_x\max_y f (x)+
y^\top\mathbf {A} xg (y) $, where $ f (x) $ and $ g (y) $ are smooth and convex functions. We …
y^\top\mathbf {A} xg (y) $, where $ f (x) $ and $ g (y) $ are smooth and convex functions. We …
Sharper rates for separable minimax and finite sum optimization via primal-dual extragradient methods
We design accelerated algorithms with improved rates for several fundamental classes of
optimization problems. Our algorithms all build upon techniques related to the analysis of …
optimization problems. Our algorithms all build upon techniques related to the analysis of …
Lifted primal-dual method for bilinearly coupled smooth minimax optimization
We study the bilinearly coupled minimax problem: $\min_ {x}\max_ {y} f (x)+ y^\top A xh (y) $,
where $ f $ and $ h $ are both strongly convex smooth functions and admit first-order …
where $ f $ and $ h $ are both strongly convex smooth functions and admit first-order …
Smooth monotone stochastic variational inequalities and saddle point problems: A survey
This paper is a survey of methods for solving smooth,(strongly) monotone stochastic
variational inequalities. To begin with, we present the deterministic foundation from which …
variational inequalities. To begin with, we present the deterministic foundation from which …
No-regret dynamics in the fenchel game: A unified framework for algorithmic convex optimization
We develop an algorithmic framework for solving convex optimization problems using no-
regret game dynamics. By converting the problem of minimizing a convex function into an …
regret game dynamics. By converting the problem of minimizing a convex function into an …
Semi-Streaming Bipartite Matching in Fewer Passes and Optimal Space∗
We provide Õ (∊–1)-pass semi-streaming algorithms for computing (1–∊)-approximate
maximum cardinality matchings in bipartite graphs. Our most efficient methods are …
maximum cardinality matchings in bipartite graphs. Our most efficient methods are …
Robust accelerated primal-dual methods for computing saddle points
We consider strongly-convex-strongly-concave saddle point problems assuming we have
access to unbiased stochastic estimates of the gradients. We propose a stochastic …
access to unbiased stochastic estimates of the gradients. We propose a stochastic …
Inexact model: A framework for optimization and variational inequalities
In this paper, we propose a general algorithmic framework for the first-order methods in
optimization in a broad sense, including minimization problems, saddle-point problems and …
optimization in a broad sense, including minimization problems, saddle-point problems and …
Regularized box-simplex games and dynamic decremental bipartite matching
Box-simplex games are a family of bilinear minimax objectives which encapsulate graph-
structured problems such as maximum flow [She17], optimal transport [JST19], and bipartite …
structured problems such as maximum flow [She17], optimal transport [JST19], and bipartite …