[BOK][B] First-order and stochastic optimization methods for machine learning

G Lan - 2020 - Springer
Since its beginning, optimization has played a vital role in data science. The analysis and
solution methods for many statistical and machine learning models rely on optimization. The …

Conditional gradient methods

G Braun, A Carderera, CW Combettes… - arxiv preprint arxiv …, 2022 - arxiv.org
The purpose of this survey is to serve both as a gentle introduction and a coherent overview
of state-of-the-art Frank--Wolfe algorithms, also called conditional gradient algorithms, for …

One sample stochastic frank-wolfe

M Zhang, Z Shen, A Mokhtari… - International …, 2020 - proceedings.mlr.press
One of the beauties of the projected gradient descent method lies in its rather simple
mechanism and yet stable behavior with inexact, stochastic gradients, which has led to its …

Conditional gradient methods via stochastic path-integrated differential estimator

A Yurtsever, S Sra, V Cevher - International Conference on …, 2019 - proceedings.mlr.press
We propose a class of variance-reduced stochastic conditional gradient methods. By
adopting the recent stochastic path-integrated differential estimator technique (SPIDER) of …

Stochastic frank-wolfe: Unified analysis and zoo of special cases

R Nazykov, A Shestakov, V Solodkin… - International …, 2024 - proceedings.mlr.press
Abstract The Conditional Gradient (or Frank-Wolfe) method is one of the most well-known
methods for solving constrained optimization problems appearing in various machine …

Stochastic conditional gradient++:(non) convex minimization and continuous submodular maximization

H Hassani, A Karbasi, A Mokhtari, Z Shen - SIAM Journal on Optimization, 2020 - SIAM
In this paper, we consider the general nonoblivious stochastic optimization where the
underlying stochasticity may change during the optimization procedure and depends on the …

Solving eigenvalue PDEs of metastable diffusion processes using artificial neural networks

W Zhang, T Li, C Schütte - Journal of Computational Physics, 2022 - Elsevier
In this paper, we consider the eigenvalue PDE problem of the infinitesimal generators of
metastable diffusion processes. We propose a numerical algorithm based on training …

Zeroth-order methods for constrained nonconvex nonsmooth stochastic optimization

Z Liu, C Chen, L Luo, BKH Low - Forty-first International Conference …, 2024 - openreview.net
This paper studies the problem of solving nonconvex nonsmooth optimization over a closed
convex set. Most previous works tackle such problems by transforming the constrained …

Constrained stochastic nonconvex optimization with state-dependent markov data

A Roy, K Balasubramanian… - Advances in neural …, 2022 - proceedings.neurips.cc
We study stochastic optimization algorithms for constrained nonconvex stochastic
optimization problems with Markovian data. In particular, we focus on the case when the …

Accelerated stochastic gradient-free and projection-free methods

F Huang, L Tao, S Chen - International conference on …, 2020 - proceedings.mlr.press
In the paper, we propose a class of accelerated stochastic gradient-free and projection-free
(aka, zeroth-order Frank-Wolfe) methods to solve the constrained stochastic and finite-sum …