NeurIPS 2025poster0 citations
Sequentially Auditing Differential Privacy
Tomás González, Mateo Dulce Rubio, Aaditya Ramdas, Mónica Ribero
Abstract
We propose a practical sequential test for auditing differential privacy guarantees of black-box mechanisms. The test processes streams of mechanisms' outputs providing anytime-valid inference while controlling Type I error, overcoming the fixed sample size limitation of previous batch auditing methods. Experiments show this test detects violations with sample sizes that are orders of magnitude smaller than existing methods, reducing this number from 50K to a few hundred examples, across diverse realistic mechanisms. Notably, it identifies DP-SGD privacy violations in \textit{under} one training run, unlike prior methods needing full model training.
Differential privacytesting by betting
BibTeX
@inproceedings{
gonzalez2025sequentially,
title={Sequentially Auditing Differential Privacy},
author={Tom{\'a}s Gonz{\'a}lez and Mateo Dulce Rubio and Aaditya Ramdas and M{\'o}nica Ribero},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=tlmKcZFAtL}
}