We provide a new analysis of local SGD, removing unnecessary assumptions and
elaborating on the difference between two data regimes: identical and heterogeneous. In …

Federated Learning (FL) is popular for communication-efficient learning from distributed
data. To utilize data at different clients without moving them to the cloud, algorithms such as …

We develop and analyze MARINA: a new communication efficient method for non-convex
distributed learning over heterogeneous datasets. MARINA employs a novel communication …

Federated Learning is a distributed learning setting where the goal is to train a centralized
model with training data distributed over a large number of heterogeneous clients, each with …

Federated Learning (FL) refers to the paradigm where multiple worker nodes (WNs) build a
joint model by using local data. Despite extensive research, for a generic non-convex FL …

We develop FedCluster-a novel federated learning framework with improved optimization
efficiency, and investigate its theoretical convergence properties. The FedCluster groups the …

Recently, local SGD has got much attention and been extensively studied in the distributed
learning community to overcome the communication bottleneck problem. However, the …

We develop and analyze DASHA: a new family of methods for nonconvex distributed
optimization problems. When the local functions at the nodes have a finite-sum or an …

The rapid development of machine learning and deep learning has introduced increasingly
complex optimization challenges that must be addressed. Indeed, training modern …

Federated averaging (FedAvg) is a well-recognized framework for distributed learning that
efficiently manages communication. Several algorithms have emerged to enhance the …