DiGradPatch: Black-Box Patch Attacks via Diffusion-Based Double Gradient and Sensitive Distribution Guidance

Abstract

Deep neural networks have demonstrated vulnerabilities to black-box adversarial patch attacks in image analysis tasks, raising concerns about their robustness in safety-critical applications. Current methods typically rely on randomized search strategies to determine patch locations and apply unrestricted pixel perturbations in the patch area, leading to high query costs and significant visual distortions that reduce imperceptibility. To address these limitations, we propose Black-Box Patch Attacks via Diffusion-Based Double Gradient and Sensitive Distribution Guidance (DiGradPatch), a novel method designed to generate highly imperceptible adversarial samples with minimal query cost. Our approach leverages the prior sensitive distribution of a surrogate model to efficiently locate the patch by maximizing the joint probability distribution between the global model prediction and selected image regions. Furthermore, we introduce the diffusion model into the black-box patch attack framework for the first time. The prior gradient from the diffusion model, in conjunction with the estimated gradient of the target model, is used to guide the direction of perturbations. This allows the generation of adversarial perturbations within the patch area that align with the distribution of natural samples, effectively reducing perceptual distortions. Extensive experiments on the ImageNet dataset show that DiGradPatch achieves superior imperceptibility with significantly reduced query costs, maintaining a high attack success rate. Compared to existing methods, our approach requires fewer than 10 queries and achieves an L<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</inf> norm that is only 1/10 of state-of-the-art techniques, while maintaining the highest attack success rate.