In recent years, Federated Learning (FL) has gained relevance in training collaborative
models without sharing sensitive data. Since its birth, Centralized FL (CFL) has been the …

In distributed optimization of multi-agent systems, agents cooperate to minimize a global
function which is a sum of local objective functions. Motivated by applications including …

We consider two federated learning algorithms for training partially personalized models,
where the shared and personal parameters are updated either simultaneously or alternately …

Adam is an adaptive gradient descent approach that is commonly used in back-propagation
(BP) algorithms for training feed-forward neural networks (FFNNs). However, it has the …

Federated averaging (FedAvg) is a communication-efficient algorithm for distributed training
with an enormous number of clients. In FedAvg, clients keep their data locally for privacy …

Most distributed machine learning systems nowadays, including TensorFlow and CNTK, are
built in a centralized fashion. One bottleneck of centralized algorithms lies on high …

Distributed stochastic gradient descent is an important subroutine in distributed learning. A
setting of particular interest is when the clients are mobile devices, where two important …

Most commonly used distributed machine learning systems are either synchronous or
centralized asynchronous. Synchronous algorithms like AllReduce-SGD perform poorly in a …

This paper considers the problem of distributed optimization over time-varying graphs. For
the case of undirected graphs, we introduce a distributed algorithm, referred to as DIGing …

There has been a growing effort in studying the distributed optimization problem over a
network. The objective is to optimize a global function formed by a sum of local functions …