Feasibility Analysis of real-time Robustness Certification

Abstract

The robustness certification of deep neural networks (DNNs) is crucial in many safety-critical domains. Randomized Smoothing (RS) has emerged as the current state-of-the-art method for DNN robustness verification that successfully scales on large DNNs used in practice, has achieved excellent results, and has been extended for a large variety of adversarial perturbation scenarios. However, an important cost in RS is during inference, since it requires passing tens or hundreds of thousands of perturbed samples through the DNN to perform the verification. In this work we aim to address this, and explore what happens as we decrease the number of samples by orders of magnitude, and the effect on the certified radius. Surprisingly, we find that the performance reduction in terms of average certified radius is not too large, even if we decrease the number of samples by two orders of magnitude, or more. Moreover, we find that the resulting certified radius reduction can be mitigated using off-the-self methods designed to improve RS performance. This can pave the way for dramatically faster robustness certification, unlocking the possibility of performing it in real-time, which we demonstrate. We perform a detailed analysis, both theoretically and experimentally, and show promising results on the standard CIFAR-10 and ImageNet datasets.