Distributed deep learning (DL) has become prevalent in recent years to reduce training time
by leveraging multiple computing devices (eg, GPUs/TPUs) due to larger models and …

Abstract Stochastic Gradient Descent (SGD) is arguably the most important single algorithm
in modern machine learning. Although SGD with unbiased gradient estimators has been …

To enable large-scale machine learning in bandwidth-hungry environments such as
wireless networks, significant progress has been made recently in designing communication …

In this work we focus our attention on distributed optimization problems in the context where
the communication time between the server and the workers is non-negligible. We obtain …

Due to the high communication overhead when training machine learning models in a
distributed environment, modern algorithms invariably rely on lossy communication …

Communication efficiency has been widely recognized as the bottleneck for large-scale
decentralized machine learning applications in multi-agent or federated environments. To …

Vertical federated learning (VFL), where data features are stored in multiple parties
distributively, is an important area in machine learning. However, the communication …

Recent advances in distributed optimization and learning have shown that communication
compression is one of the most effective means of reducing communication. While there …

Inspired by a recent breakthrough of Mishchenko et al.[2022], who for the first time showed
that local gradient steps can lead to provable communication acceleration, we propose an …

Federated learning (FL) is an emerging distributed machine learning paradigm that aims to
realize model training without gathering the data from data sources to a central processing …