Uniform stability is a notion of algorithmic stability that bounds the worst case change in the
model output by the algorithm when a single data point in the dataset is replaced. An …

The field of deep learning has witnessed significant progress in recent times, particularly in
areas such as computer vision (CV), natural language processing (NLP), and speech. The …

The\FedProx~ algorithm is a simple yet powerful distributed proximal point optimization
method widely used for federated learning (FL) over heterogeneous data. Despite its …

The adversarial training is a popular tool to remedy the vulnerability of deep learning models
against adversarial attacks, and there is rich theoretical literature on the training loss of …

We study the generalization properties of the popular stochastic optimization method known
as stochastic gradient descent (SGD) for optimizing general non-convex loss functions. Our …

This paper studies the algorithmic stability and generalizability of decentralized stochastic
gradient descent (D-SGD). We prove that the consensus model learned by D-SGD is …

The training and generalization dynamics of the Transformer's core mechanism, namely the
Attention mechanism, remain under-explored. Besides, existing analyses primarily focus on …

Heavy-tail phenomena in stochastic gradient descent (SGD) have been reported in several
empirical studies. Experimental evidence in previous works suggests a strong interplay …

The generalization ability often determines the success of machine learning algorithms in
practice. Therefore, it is of great theoretical and practical importance to understand and …

Multi-objective learning (MOL) often arises in emerging machine learning problems when
multiple learning criteria or tasks need to be addressed. Recent works have developed …