In this work we focus our attention on distributed optimization problems in the context where
the communication time between the server and the workers is non-negligible. We obtain …

Decentralized stochastic gradient descent (D-SGD) allows collaborative learning on
massive devices simultaneously without the control of a central server. However, existing …

Injecting noise within gradient descent has several desirable features, such as smoothing
and regularizing properties. In this paper, we investigate the effects of injecting noise before …

Differentially private learning algorithms inject noise into the learning process. While the
most common private learning algorithm, DP-SGD, adds independent Gaussian noise in …

We study gradient descent under linearly correlated noise. Our work is motivated by recent
practical methods for optimization with differential privacy (DP), such as DP-FTRL, which …

Sharpness-Aware Minimization (SAM) is most known for achieving state-of the-art
performances on natural image and language tasks. However, its most pronounced …

This paper proposes a flexible probabilistic activation function that enhances the training
and operation of artificial neural networks by intentionally injecting noise to gain additional …

It is well known that the finite step-size ($ h $) in Gradient Descent (GD) implicitly regularizes
solutions to flatter minima. A natural question to ask is" Does the momentum parameter …

Federated edge learning is envisioned as the bedrock of enabling intelligence in next-
generation wireless networks, but the limited spectral resources often constrain its …

Knowledge Distillation (KD) is a commonly used technique for improving the generalization
of compact Pre-trained Language Models (PLMs) on downstream tasks. However, such …