Federated learning is a distributed framework according to which a model is trained over a
set of devices, while kee** data localized. This framework faces several systems-oriented …

Nowadays, the training data and neural network models are getting increasingly large. The
training time of deep learning will become unbearably long on a single machine. To reduce …

This paper proposes ElasticFlow, an elastic serverless training platform for distributed deep
learning. ElasticFlow provides a serverless interface with two distinct features:(i) users …

We consider the problem of decentralized consensus optimization, where the sum of n
smooth and strongly convex functions are minimized over n distributed agents that form a …

We consider a decentralized learning problem, where a set of computing nodes aim at
solving a non-convex optimization problem collaboratively. It is well-known that …

In this article, we consider decentralized optimization problems where agents have
individual cost functions to minimize subject to subspace constraints that require the …

To address the big data challenges, serverless multi-party collaborative training has recently
attracted attention in the data mining community, since they can cut down the …

Modern distributed training of machine learning models often suffers from high
communication overhead for synchronizing stochastic gradients and model parameters. In …

This paper studies distributed algorithms for (strongly convex) composite optimization
problems over mesh networks, subject to quantized communications. Instead of focusing on …

Communication has been considered as a major bottleneck in large-scale decentralized
training systems since participating nodes iteratively exchange large amounts of …