Abstract Machine Learning (ML) models are ubiquitous in real-world applications and are a
constant focus of research. Modern ML models have become more complex, deeper, and …

Federated learning (FL) is a machine learning setting where many clients (eg, mobile
devices or whole organizations) collaboratively train a model under the orchestration of a …

Several recently proposed stochastic optimization methods that have been successfully
used in training deep networks such as RMSProp, Adam, Adadelta, Nadam are based on …

Federated learning is a distributed machine learning paradigm in which a large number of
clients coordinate with a central server to learn a model without sharing their own training …

Most popular optimizers for deep learning can be broadly categorized as adaptive methods
(eg~ Adam) and accelerated schemes (eg~ stochastic gradient descent (SGD) with …

Decentralised learning is attracting more and more interest because it embodies the
principles of data minimisation and focused data collection, while favouring the transparency …

This monograph portrays optimization as a process. In many practical applications the
environment is so complex that it is infeasible to lay out a comprehensive theoretical model …

We present a new family of subgradient methods that dynamically incorporate knowledge of
the geometry of the data observed in earlier iterations to perform more informative gradient …

Adaptive optimization methods such as AdaGrad, RMSprop and Adam have been proposed
to achieve a rapid training process with an element-wise scaling term on learning rates …

Adaptive optimization methods, which perform local optimization with a metric constructed
from the history of iterates, are becoming increasingly popular for training deep neural …