Distributed learning is envisioned as the bedrock of next-generation intelligent networks,
where intelligent agents, such as mobile devices, robots, and sensors, exchange information …

Deep learning has recently become very popular on account of its incredible success in
many complex datadriven applications, including image classification and speech …

The growing energy and performance costs of deep learning have driven the community to
reduce the size of neural networks by selectively pruning components. Similarly to their …

Federated learning (FL) is a machine learning setting where many clients (eg, mobile
devices or whole organizations) collaboratively train a model under the orchestration of a …

Numerous machine learning models can be formulated as a stochastic minimax optimization
problem, such as imbalanced data classification with AUC maximization. Develo** …

Federated Learning is a machine learning scheme in which a shared prediction model can
be collaboratively learned by a number of distributed nodes using their locally stored data. It …

Error feedback (EF), also known as error compensation, is an immensely popular
convergence stabilization mechanism in the context of distributed training of supervised …

Distributed training of foundation models, especially large language models (LLMs), is
communication-intensive and so has heavily relied on centralized data centers with fast …

Decentralized federated learning, inherited from decentralized learning, enables the edge
devices to collaborate on model training in a peer-to-peer manner without the assistance of …

Abstract Stochastic Gradient Descent (SGD) is arguably the most important single algorithm
in modern machine learning. Although SGD with unbiased gradient estimators has been …