Momentum and stochastic momentum for stochastic gradient, newton, proximal point and subspace descent methods

N Loizou, P Richtárik - Computational Optimization and Applications, 2020 - Springer
In this paper we study several classes of stochastic optimization algorithms enriched with
heavy ball momentum. Among the methods studied are: stochastic gradient descent …

A stochastic trust region algorithm based on careful step normalization

FE Curtis, K Scheinberg, R Shi - Informs Journal on …, 2019 - pubsonline.informs.org
An algorithm is proposed for solving stochastic and finite-sum minimization problems. Based
on a trust region methodology, the algorithm employs normalized steps, at least as long as …

Stochastic Nash equilibrium problems: Models, analysis, and algorithms

J Lei, UV Shanbhag - IEEE Control Systems Magazine, 2022 - ieeexplore.ieee.org
Decision making under uncertainty has been studied extensively over the last 70 years, if
not earlier. In the field of optimization, models for two-stage, stochastic, linear programming …

Linearly convergent variable sample-size schemes for stochastic Nash games: Best-response schemes and distributed gradient-response schemes

J Lei, UV Shanbhag - 2018 IEEE Conference on Decision and …, 2018 - ieeexplore.ieee.org
This paper considers an N-player stochastic Nash game in which the i th player minimizes a
composite objective fi (x)+ ri (xi), where fi is expectation-valued and ri has an efficient prox …

Constrained and composite optimization via adaptive sampling methods

Y **e, R Bollapragada, R Byrd… - IMA Journal of …, 2024 - academic.oup.com
The motivation for this paper stems from the desire to develop an adaptive sampling method
for solving constrained optimization problems, in which the objective function is stochastic …

Complexity guarantees for an implicit smoothing-enabled method for stochastic MPECs

S Cui, UV Shanbhag, F Yousefian - Mathematical Programming, 2023 - Springer
Mathematical programs with equilibrium constraints (MPECs) represent a class of
hierarchical programs that allow for modeling problems in engineering, economics, finance …

A variable sample-size stochastic quasi-Newton method for smooth and nonsmooth stochastic convex optimization

A Jalilzadeh, A Nedić, UV Shanbhag… - Mathematics of …, 2022 - pubsonline.informs.org
Classical theory for quasi-Newton schemes has focused on smooth, deterministic,
unconstrained optimization, whereas recent forays into stochastic convex optimization have …

Asynchronous schemes for stochastic and misspecified potential games and nonconvex optimization

J Lei, UV Shanbhag - Operations Research, 2020 - pubsonline.informs.org
The distributed computation of equilibria and optima has seen growing interest in a broad
collection of networked problems. We consider the computation of Nash equilibria of convex …

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

S Cui, U Shanbhag, M Staudigl, P Vuong - … Optimization and Applications, 2022 - Springer
We consider monotone inclusions defined on a Hilbert space where the operator is given by
the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz …

On the computation of equilibria in monotone and potential stochastic hierarchical games

S Cui, UV Shanbhag - Mathematical Programming, 2023 - Springer
We consider a class of noncooperative hierarchical N-player games where the i th player
solves a parametrized stochastic mathematical program with equilibrium constraints (MPEC) …