Optimization is a critical component in deep learning. We think optimization for neural
networks is an interesting topic for theoretical research due to various reasons. First, its …

Federated learning is a key scenario in modern large-scale machine learning where the
data remains distributed over a large number of clients and the task is to learn a centralized …

Federated learning involves training statistical models over remote devices or siloed data
centers, such as mobile phones or hospitals, while kee** data localized. Training in …

Federated Learning is a distributed learning paradigm with two key challenges that
differentiate it from traditional distributed optimization:(1) significant variability in terms of the …

Multiple medical institutions collaboratively training a model using federated learning (FL)
has become a promising solution for maximizing the potential of data-driven models, yet the …

Ever since\citet {reddi2019convergence} pointed out the divergence issue of Adam, many
new variants have been designed to obtain convergence. However, vanilla Adam remains …

We propose a general yet simple theorem describing the convergence of SGD under the
arbitrary sampling paradigm. Our theorem describes the convergence of an infinite array of …

The success of deep learning is due, to a large extent, to the remarkable effectiveness of
gradient-based optimization methods applied to large neural networks. The purpose of this …

Federated learning (FL) is a challenging setting for optimization due to the heterogeneity of
the data across different clients which gives rise to the client drift phenomenon. In fact …

Multi-Task Learning (MTL) has achieved success in various fields. However, how to balance
different tasks to achieve good performance is a key problem. To achieve the task balancing …