Bilevel optimization, the problem of minimizing a value function which involves the arg-
minimum of another function, appears in many areas of machine learning. In a large scale …

This paper proposes a new algorithm--the\underline {S} ingle-timescale Do\underline {u} ble-
momentum\underline {St} ochastic\underline {A} pprox\underline {i} matio\underline …

Many problems in machine learning involve bilevel optimization (BLO), including
hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems …

Abstract Differentiable ARchiTecture Search (DARTS) has recently become the mainstream
in the neural architecture search (NAS) due to its efficiency and simplicity. With a gradient …

We study a class of algorithms for solving bilevel optimization problems in both stochastic
and deterministic settings when the inner-level objective is strongly convex. Specifically, we …

The goal of coreset selection in supervised learning is to produce a weighted subset of data,
so that training only on the subset achieves similar performance as training on the entire …

Finding the optimal hyperparameters of a model can be cast as a bilevel optimization
problem, typically solved using zero-order techniques. In this work we study first-order …

In this paper, we develop an approximation scheme for solving bilevel programs with
equilibrium constraints, which are generally difficult to solve. Among other things, calculating …

Estimating hyperparameters has been a long-standing problem in machine learning. We
consider the case where the task at hand is modeled as the solution to an optimization …

We analyse a general class of bilevel problems, in which the upper-level problem consists in
the minimization of a smooth objective function and the lower-level problem is to find the …