A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation, and Blackwell's Theorem

WJ Su - Annual Review of Statistics and Its Application, 2024 - annualreviews.org
Differential privacy is widely considered the formal privacy for privacy-preserving data
analysis due to its robust and rigorous guarantees, with increasingly broad adoption in …

The last iterate advantage: Empirical auditing and principled heuristic analysis of differentially private sgd

T Steinke, M Nasr, A Ganesh, B Balle… - arxiv preprint arxiv …, 2024 - arxiv.org
We propose a simple heuristic privacy analysis of noisy clipped stochastic gradient descent
(DP-SGD) in the setting where only the last iterate is released and the intermediate iterates …

Convergent Differential Privacy Analysis for General Federated Learning: the -DP Perspective

Y Sun, L Shen, D Tao - arxiv preprint arxiv:2408.15621, 2024 - arxiv.org
Federated learning (FL) is an efficient collaborative training paradigm extensively developed
with a focus on local privacy, and differential privacy (DP) is a classical approach to capture …

Shifted Composition III: Local Error Framework for KL Divergence

JM Altschuler, S Chewi - arxiv preprint arxiv:2412.17997, 2024 - arxiv.org
Coupling arguments are a central tool for bounding the deviation between two stochastic
processes, but traditionally have been limited to Wasserstein metrics. In this paper, we apply …

Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios

S Yoon, W Jeung, A No - arxiv preprint arxiv:2412.01756, 2024 - arxiv.org
Auditing Differentially Private Stochastic Gradient Descent (DP-SGD) in the final model
setting is challenging and often results in empirical lower bounds that are significantly looser …

The 2020 United States Decennial Census Is More Private Than You (Might) Think

B Su, WJ Su, C Wang - arxiv preprint arxiv:2410.09296, 2024 - arxiv.org
The US Decennial Census serves as the foundation for many high-profile policy decision-
making processes, including federal funding allocation and redistricting. In 2020, the …

Approximating Two-Layer ReLU Networks for Hidden State Analysis in Differential Privacy

A Koskela - arxiv preprint arxiv:2407.04884, 2024 - arxiv.org
The hidden state threat model of differential privacy (DP) assumes that the adversary has
access only to the final trained machine learning (ML) model, without seeing intermediate …

The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD

M Nasr, T Steinke, B Balle, CA Choquette-Choo… - 2024 - openreview.net
We propose a simple heuristic privacy analysis of noisy clipped stochastic gradient descent
(DP-SGD) in the setting where only the last iterate is released and the intermediate iterates …

Neural collapse meets differential privacy: curious behaviors of NoisyGD with near-perfect representation learning

C Wang, Y Zhu, WJ Su, YX Wang - arxiv preprint arxiv:2405.08920, 2024 - arxiv.org
A recent study by De et al.(2022) has reported that large-scale representation learning
through pre-training on a public dataset significantly enhances differentially private (DP) …

Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite losses

W Kong, M Ribero - arxiv preprint arxiv:2407.05237, 2024 - arxiv.org
Differentially-private stochastic gradient descent (DP-SGD) is a family of iterative machine
learning training algorithms that privatize gradients to generate a sequence of differentially …