We study the SAM (Sharpness-Aware Minimization) optimizer which has recently attracted a
lot of interest due to its increased performance over more classical variants of stochastic …

Despite the vast empirical evidence supporting the efficacy of adaptive optimization methods
in deep learning, their theoretical understanding is far from complete. This work introduces …

Minimax optimization problems have attracted a lot of attention over the past few years, with
applications ranging from economics to machine learning. While advanced optimization …

We give quantitative estimates for the rate of convergence of Riemannian stochastic
gradient descent (RSGD) to Riemannian gradient flow and to a diffusion process, the so …

Stochastic Gradient Descent (SGD) and its variants are almost universally used to train
neural networks and to fit a variety of other parametric models. An important hyperparameter …

Deep learning technologies are skyrocketing in popularity across a wide range of domains,
with groundbreaking accomplishments in fields such as natural language processing …