Constant matters: Fine-grained error bound on differentially private continual observation

H Fichtenberger, M Henzinger… - … on Machine Learning, 2023 - proceedings.mlr.press
We study fine-grained error bounds for differentially private algorithms for counting under
continual observation. Our main insight is that the matrix mechanism when using lower …

Differentially private attention computation

Y Gao, Z Song, X Yang, Y Zhou - arxiv preprint arxiv:2305.04701, 2023 - arxiv.org
Large language models (LLMs), especially those based on the Transformer architecture,
have had a profound impact on various aspects of daily life, such as natural language …

Fast quantum algorithm for attention computation

Y Gao, Z Song, X Yang, R Zhang - arxiv preprint arxiv:2307.08045, 2023 - arxiv.org
Large language models (LLMs) have demonstrated exceptional performance across a wide
range of tasks. These models, powered by advanced deep learning techniques, have …

Randomized and deterministic attention sparsification algorithms for over-parameterized feature dimension

Y Deng, S Mahadevan, Z Song - arxiv preprint arxiv:2304.04397, 2023 - arxiv.org
Large language models (LLMs) have shown their power in different areas. Attention
computation, as an important subroutine of LLMs, has also attracted interests in theory …

Counting distinct elements in the turnstile model with differential privacy under continual observation

P Jain, I Kalemaj, S Raskhodnikova… - Advances in …, 2023 - proceedings.neurips.cc
Privacy is a central challenge for systems that learn from sensitive data sets, especially
when a system's outputs must be continuously updated to reflect changing data. We …

A unifying framework for differentially private sums under continual observation

M Henzinger, J Upadhyay, S Upadhyay - … of the 2024 Annual ACM-SIAM …, 2024 - SIAM
We study the problem of maintaining a differentially private decaying sum under continual
observation. We give a unifying framework and an efficient algorithm for this problem for any …

Constant matters: Fine-grained Complexity of Differentially Private Continual Observation

H Fichtenberger, M Henzinger, J Upadhyay - arxiv preprint arxiv …, 2022 - arxiv.org
We study fine-grained error bounds for differentially private algorithms for counting under
continual observation. Our main insight is that the matrix mechanism when using lower …

How to make your approximation algorithm private: A black-box differentially-private transformation for tunable approximation algorithms of functions with low …

J Blocki, E Grigorescu, T Mukherjee, S Zhou - arxiv preprint arxiv …, 2022 - arxiv.org
We develop a framework for efficiently transforming certain approximation algorithms into
differentially-private variants, in a black-box manner. Specifically, our results focus on …

Improved differentially private continual observation using group algebra

M Henzinger, J Upadhyay - Proceedings of the 2025 Annual ACM-SIAM …, 2025 - SIAM
Differentially private weighted prefix sum under continual observation is a crucial component
in the production-level deployment of private next-word prediction for Gboard, which …

Private counting of distinct elements in the turnstile model and extensions

M Henzinger, AR Sricharan, TA Steiner - arxiv preprint arxiv:2408.11637, 2024 - arxiv.org
Privately counting distinct elements in a stream is a fundamental data analysis problem with
many applications in machine learning. In the turnstile model, Jain et al.[NeurIPS2023] …