How to dp-fy ml: A practical guide to machine learning with differential privacy
Abstract Machine Learning (ML) models are ubiquitous in real-world applications and are a
constant focus of research. Modern ML models have become more complex, deeper, and …
constant focus of research. Modern ML models have become more complex, deeper, and …
Differentially private natural language models: Recent advances and future directions
Recent developments in deep learning have led to great success in various natural
language processing (NLP) tasks. However, these applications may involve data that …
language processing (NLP) tasks. However, these applications may involve data that …
How to Protect Copyright Data in Optimization of Large Language Models?
The softmax operator is a crucial component of large language models (LLMs), which have
played a transformative role in computer research. Due to the centrality of the softmax …
played a transformative role in computer research. Due to the centrality of the softmax …
(Amplified) Banded Matrix Factorization: A unified approach to private training
Matrix factorization (MF) mechanisms for differential privacy (DP) have substantially
improved the state-of-the-art in privacy-utility-computation tradeoffs for ML applications in a …
improved the state-of-the-art in privacy-utility-computation tradeoffs for ML applications in a …
On the convergence of federated averaging with cyclic client participation
Abstract Federated Averaging (FedAvg) and its variants are the most popular optimization
algorithms in federated learning (FL). Previous convergence analyses of FedAvg either …
algorithms in federated learning (FL). Previous convergence analyses of FedAvg either …
Private distribution learning with public data: The view from sample compression
We study the problem of private distribution learning with access to public data. In this setup,
which we refer to as* public-private learning*, the learner is given public and private …
which we refer to as* public-private learning*, the learner is given public and private …
Can Public Large Language Models Help Private Cross-device Federated Learning?
We study (differentially) private federated learning (FL) of language models. The language
models in cross-device FL are relatively small, which can be trained with meaningful formal …
models in cross-device FL are relatively small, which can be trained with meaningful formal …
Fine-tuning large language models with user-level differential privacy
We investigate practical and scalable algorithms for training large language models (LLMs)
with user-level differential privacy (DP) in order to provably safeguard all the examples …
with user-level differential privacy (DP) in order to provably safeguard all the examples …
One-shot empirical privacy estimation for federated learning
Privacy estimation techniques for differentially private (DP) algorithms are useful for
comparing against analytical bounds, or to empirically measure privacy loss in settings …
comparing against analytical bounds, or to empirically measure privacy loss in settings …
Efficient and near-optimal noise generation for streaming differential privacy
In the task of differentially private (DP) continual counting, we receive a stream of increments
and our goal is to output an approximate running total of these increments, without revealing …
and our goal is to output an approximate running total of these increments, without revealing …