Automatic clip**: Differentially private deep learning made easier and stronger
Per-example gradient clip** is a key algorithmic step that enables practical differential
private (DP) training for deep learning models. The choice of clip** threshold $ R …
private (DP) training for deep learning models. The choice of clip** threshold $ R …
Convergence of adagrad for non-convex objectives: Simple proofs and relaxed assumptions
We provide a simple convergence proof for AdaGrad optimizing non-convex objectives
under only affine noise variance and bounded smoothness assumptions. The proof is …
under only affine noise variance and bounded smoothness assumptions. The proof is …
Generalized-smooth nonconvex optimization is as efficient as smooth nonconvex optimization
Various optimal gradient-based algorithms have been developed for smooth nonconvex
optimization. However, many nonconvex machine learning problems do not belong to the …
optimization. However, many nonconvex machine learning problems do not belong to the …
DPSUR: accelerating differentially private stochastic gradient descent using selective update and release
J Fu, Q Ye, H Hu, Z Chen, L Wang, K Wang… - ar** for non-convex optimization
A Reisizadeh, H Li, S Das, A Jadbabaie - ar** is a standard training technique used in deep learning applications such
as large-scale language modeling to mitigate exploding gradients. Recent experimental …
as large-scale language modeling to mitigate exploding gradients. Recent experimental …
A theory to instruct differentially-private learning via clip** bias reduction
H ** and communication compression
Achieving communication efficiency in decentralized machine learning has been attracting
significant attention, with communication compression recognized as an effective technique …
significant attention, with communication compression recognized as an effective technique …