Bi-Level Optimization (BLO) is originated from the area of economic game theory and then
introduced into the optimization community. BLO is able to handle problems with a …

This paper analyzes a two-timescale stochastic algorithm framework for bilevel optimization.
Bilevel optimization is a class of problems which exhibits a two-level structure, and its goal is …

Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in
learning fields. The validity of existing works heavily rely on either a restrictive Lower-Level …

Stochastic nested optimization, including stochastic compositional, min-max, and bilevel
optimization, is gaining popularity in many machine learning applications. While the three …

Bilevel optimization (BO) is useful for solving a variety of important machine learning
problems including but not limited to hyperparameter optimization, meta-learning, continual …

Standard federated optimization methods successfully apply to stochastic problems with
single-level structure. However, many contemporary ML problems-including adversarial …

Bilevel optimization has been widely applied in many important machine learning
applications such as hyperparameter optimization and meta-learning. Recently, several …

We consider stochastic unconstrained bilevel optimization problems when only the first-
order gradient oracles are available. While numerous optimization methods have been …

Coresets are small data summaries that are sufficient for model training. They can be
maintained online, enabling efficient handling of large data streams under resource …

This paper proposes a new algorithm--the\underline {S} ingle-timescale Do\underline {u} ble-
momentum\underline {St} ochastic\underline {A} pprox\underline {i} matio\underline …