The Test of Tests: A Framework for Differentially Private Hypothesis Testing

Zeki Kazan; Kaiyan Shi; Adam Groce; Andrew Bray

ICML 2023poster14 citations

The Test of Tests: A Framework for Differentially Private Hypothesis Testing

Zeki Kazan, Kaiyan Shi, Adam Groce, Andrew Bray

Abstract

We present a generic framework for creating differentially private versions of any hypothesis test in a black-box way. We analyze the resulting tests analytically and experimentally. Most crucially, we show good practical performance for small data sets, showing that at ε = 1 we only need 5-6 times as much data as in the fully public setting. We compare our work to the one existing framework of this type, as well as to several individually-designed private hypothesis tests. Our framework is higher power than other generic solutions and at least competitive with (and often better than) individually-designed tests.

BibTeX

@inproceedings{icml2023_thetestoftestsaf,
  title = {The Test of Tests: A Framework for Differentially Private Hypothesis Testing},
  author = {Zeki Kazan and Kaiyan Shi and Adam Groce and Andrew Bray},
  booktitle = {ICML 2023},
  year = {2023}
}