[BOK][B] Modern nonconvex nondifferentiable optimization

Y Cui, JS Pang - 2021 - SIAM
Mathematical optimization has always been at the heart of engineering, statistics, and
economics. In these applied domains, optimization concepts and methods have often been …

Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

M Wang, EX Fang, H Liu - Mathematical Programming, 2017 - Springer
Classical stochastic gradient methods are well suited for minimizing expected-value
objective functions. However, they do not apply to the minimization of a nonlinear function …

A single timescale stochastic approximation method for nested stochastic optimization

S Ghadimi, A Ruszczynski, M Wang - SIAM Journal on Optimization, 2020 - SIAM
We study constrained nested stochastic optimization problems in which the objective
function is a composition of two smooth functions whose exact values and derivatives are …

Accelerating stochastic composition optimization

M Wang, J Liu, EX Fang - Journal of Machine Learning Research, 2017 - jmlr.org
We consider the stochastic nested composition optimization problem where the objective is
a composition of two expected-value functions. We propose a new stochastic first-order …

Finite-sum composition optimization via variance reduced gradient descent

X Lian, M Wang, J Liu - Artificial Intelligence and Statistics, 2017 - proceedings.mlr.press
The stochastic composition optimization proposed recently by Wang et al.[2014] minimizes
the objective with the composite expectation form: $\min_x (\mathbbE_iF_i∘\mathbbE_j …

Stochastic multilevel composition optimization algorithms with level-independent convergence rates

K Balasubramanian, S Ghadimi, A Nguyen - SIAM Journal on Optimization, 2022 - SIAM
In this paper, we study smooth stochastic multilevel composition optimization problems,
where the objective function is a nested composition of T functions. We assume access to …

A stochastic subgradient method for distributionally robust non-convex and non-smooth learning

M Gürbüzbalaban, A Ruszczyński, L Zhu - Journal of Optimization Theory …, 2022 - Springer
We consider a distributionally robust formulation of stochastic optimization problems arising
in statistical learning, where robustness is with respect to ambiguity in the underlying data …

Multilevel stochastic gradient methods for nested composition optimization

S Yang, M Wang, EX Fang - SIAM Journal on Optimization, 2019 - SIAM
Stochastic gradient methods are scalable for solving large-scale optimization problems that
involve empirical expectations of loss functions. Existing results mainly apply to optimization …