The limits and potentials of local sgd for distributed heterogeneous learning with intermittent communication

KK Patel, M Glasgow, A Zindari… - The Thirty Seventh …, 2024 - proceedings.mlr.press
Local SGD is a popular optimization method in distributed learning, often outperforming mini-
batch SGD. Despite this practical success, proving the efficiency of local SGD has been …

Federated learning under periodic client participation and heterogeneous data: A new communication-efficient algorithm and analysis

M Crawshaw, M Liu - Advances in Neural Information …, 2025 - proceedings.neurips.cc
In federated learning, it is common to assume that clients are always available to participate
in training, which may not be feasible with user devices in practice. Recent works analyze …

Federated online and bandit convex optimization

KK Patel, L Wang, A Saha… - … Conference on Machine …, 2023 - proceedings.mlr.press
We study the problems of distributed online and bandit convex optimization against an
adaptive adversary. We aim to minimize the average regret on $ M $ machines working in …

Delta: Diverse client sampling for fasting federated learning

L Wang, YX Guo, T Lin, X Tang - Advances in Neural …, 2023 - proceedings.neurips.cc
Partial client participation has been widely adopted in Federated Learning (FL) to reduce the
communication burden efficiently. However, an inadequate client sampling scheme can lead …

Spam: Stochastic proximal point method with momentum variance reduction for non-convex cross-device federated learning

A Karagulyan, E Shulgin, A Sadiev… - arxiv preprint arxiv …, 2024 - arxiv.org
Cross-device training is a crucial subfield of federated learning, where the number of clients
can reach into the billions. Standard approaches and local methods are prone to issues …

Fedbcgd: Communication-efficient accelerated block coordinate gradient descent for federated learning

J Liu, F Shang, Y Liu, H Liu, Y Li, YX Gong - Proceedings of the 32nd …, 2024 - dl.acm.org
Although Federated Learning has been widely studied in recent years, there are still high
overhead expenses in each communication round for large-scale models such as Vision …

On the still unreasonable effectiveness of federated averaging for heterogeneous distributed learning

KK Patel, M Glasgow, L Wang, N Joshi… - … Learning and Analytics …, 2023 - openreview.net
Federated Averaging/local SGD is the most common optimization method for federated
learning that has proven effective in many real-world applications, dominating simple …