Miguel Angel Gonzalez Ballester

DP-KFC: Data-Free Preconditioning for Privacy-Preserving Deep Learning

Differentially private optimization suffers from a fundamental geometric mismatch: deep networks have highly anisotropic loss landscapes, yet DP-SGD injects isotropic noise. Second-order preconditioning can resolve this, but estimating curvature typically requires private data (consuming privacy bud…