Abstract Machine Learning (ML) models are ubiquitous in real-world applications and are a
constant focus of research. Modern ML models have become more complex, deeper, and …

We investigate practical and scalable algorithms for training large language models (LLMs)
with user-level differential privacy (DP) in order to provably safeguard all the examples …

In the task of differentially private (DP) continual counting, we receive a stream of increments
and our goal is to output an approximate running total of these increments, without revealing …

Differentially private learning algorithms inject noise into the learning process. While the
most common private learning algorithm, DP-SGD, adds independent Gaussian noise in …

When large language models are trained on private data, it can be a significant privacy risk
for them to memorize and regurgitate sensitive information. In this work, we propose a new …

Cross-device federated learning (FL) is a technique that trains a model on data distributed
across typically millions of edge devices without data leaving the devices. SGD is the …

Collaborative machine learning (ML) is widely used to enable institutions to learn better
models from distributed data. While collaborative approaches to learning intuitively protect …

Federated Learning (FL) is a machine learning technique that enables multiple entities to
collaboratively learn a shared model without exchanging their local data. Over the past …

Privacy amplification exploits randomness in data selection to provide tighter differential
privacy (DP) guarantees. This analysis is key to DP-SGD's success in machine learning, but …

Current state-of-the-art methods for differentially private model training are based on matrix
factorization techniques. However, these methods suffer from high computational overhead …