Federated learning is a privacy-preserving machine learning technique to train intelligent
models from decentralized data, which enables exploiting private data by communicating …

Abstract One-shot Federated Learning (FL) has recently emerged as a promising approach,
which allows the central server to learn a model in a single communication round. Despite …

In this work, we propose a communication-efficient parameterization, FedPara, for federated
learning (FL) to overcome the burdens on frequent model uploads and downloads. Our …

We develop a variational framework to understand the properties of functions learned by
fitting deep neural networks with rectified linear unit (ReLU) activations to data. We propose …

The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon,
and Mistral, provides diverse options for AI practitioners and researchers. However, most …

Deep learning (DL) has been wildly successful in practice, and most of the state-of-the-art
machine learning methods are based on neural networks (NNs). Lacking, however, is a …

Neural networks have achieved tremendous success in a large variety of applications.
However, their memory footprint and computational demand can render them impractical in …

In training of modern large natural language processing (NLP) models, it has become a
common practice to split models using 3D parallelism to multiple GPUs. Such technique …

Abstract In Federated Learning (FL), a common approach for aggregating local solutions
across clients is periodic full model averaging. It is, however, known that different layers of …

One underlying assumption of recent federated learning (FL) paradigms is that all local
models usually share the same network architecture and size, which becomes impractical …