Abstract Stochastic Gradient Descent (SGD) is arguably the most important single algorithm
in modern machine learning. Although SGD with unbiased gradient estimators has been …

In the last few years, various communication compression techniques have emerged as an
indispensable tool hel** to alleviate the communication bottleneck in distributed learning …

To enable large-scale machine learning in bandwidth-hungry environments such as
wireless networks, significant progress has been made recently in designing communication …

Due to the high communication overhead when training machine learning models in a
distributed environment, modern algorithms invariably rely on lossy communication …

Communication efficiency has been widely recognized as the bottleneck for large-scale
decentralized machine learning applications in multi-agent or federated environments. To …

In this work we focus our attention on distributed optimization problems in the context where
the communication time between the server and the workers is non-negligible. We obtain …

Many compression techniques have been proposed to reduce the communication overhead
of Federated Learning training procedures. However, these are typically designed for …

In practical federated learning (FL) systems, the communication cost between the clients and
the central server can often be a bottleneck. In this paper, we focus on biased gradient …

Communication compression, a technique aiming to reduce the information volume to be
transmitted over the air, has gained great interests in Federated Learning (FL) for the …

Recent advances in distributed optimization and learning have shown that communication
compression is one of the most effective means of reducing communication. While there …