The Internet of Things (IoT) is on the verge of a major paradigm shift. In the IoT system of the
future, IoFT, the “cloud” will be substituted by the “crowd” where model training is brought to …

Scalable and efficient distributed learning is one of the main driving forces behind the recent
rapid advancement of machine learning and artificial intelligence. One prominent feature of …

Regularization in the optimization of deep neural networks is often critical to avoid
undesirable over-fitting leading to better generalization of model. One of the most popular …

In this paper we consider parallelization for applications whose objective can be expressed
as maximizing a non-monotone submodular function under a cardinality constraint. Our …

Block coordinate descent (BCD) methods are widely used for large-scale numerical
optimization because of their cheap iteration costs, low memory requirements, amenability to …

High-quality machine learning models are dependent on access to high-quality training
data. When the data are not already available, it is tedious and costly to obtain them. Data …

Abstract Stochastic Gradient Descent or SGD is the most popular optimization algorithm for
large-scale problems. SGD estimates the gradient by uniform sampling with sample size …

Adam is a widely used optimization method for training deep learning models. It computes
individual adaptive learning rates for different parameters. In this paper, we propose a …

We consider the stochastic gradient descent (SGD) algorithm driven by a general stochastic
sequence, including iid noise and random walk on an arbitrary graph, among others; and …

We consider the problem of a Parameter Server (PS) that wishes to learn a model that fits
data distributed on the nodes of a graph. We focus on Federated Learning (FL) as a …