In distributed optimization of multi-agent systems, agents cooperate to minimize a global
function which is a sum of local objective functions. Motivated by applications including …

Since their inception, computational homogenization methods based on the fast Fourier
transform (FFT) have grown in popularity, establishing themselves as a powerful tool …

The vast majority of successful deep neural networks are trained using variants of stochastic
gradient descent (SGD) algorithms. Recent attempts to improve SGD can be broadly …

We show that parametric models trained by a stochastic gradient method (SGM) with few
iterations have vanishing generalization error. We prove our results by arguing that SGM is …

A large number of imaging problems reduce to the optimization of a cost function, with
typical structural properties. The aim of this paper is to describe the state of the art in …

We derive a second-order ordinary differential equation (ODE) which is the limit of
Nesterov's accelerated gradient method. This ODE exhibits approximate equivalence to …

Accelerated gradient methods play a central role in optimization, achieving optimal rates in
many settings. Although many generalizations and extensions of Nesterov's original …

Gradient-based optimization algorithms can be studied from the perspective of limiting
ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not …

Abstract We propose Federated Accelerated Stochastic Gradient Descent (FedAc), a
principled acceleration of Federated Averaging (FedAvg, also known as Local SGD) for …

Motivated by applications in Optimization, Game Theory, and the training of Generative
Adversarial Networks, the convergence properties of first order methods in min-max …