Zero-shot Quantization for Large-kernels via Shape-based Distribution and Diversity Self-distillation

Abstract

Zero-shot quantization (ZSQ) has emerged as an effective method to reduce model complexity and memory footprint without using original training data, thereby mitigating data privacy and security concerns during model deployment. Recently, Large-Kernel Convolutional Neural Networks (LKCNNs) have achieved state-of-the-art performance on various vision tasks, which introduce challenges in terms of increased parameters and network complexity, making them difficult to deploy on resource-constrained edge devices. Despite the success of ZSQ, existing methods fail to apply to LKCNNs due to architectural differences such as Batch Normalization (BN) layers in models and thus result in significant performance declines. In this paper, we propose a novel ZSQ framework tailored specifically for LKCNNs, considering their two key characteristics: the large receptive field and the reliance on shape bias. Correspondingly, we first employ an edge detection-based loss to optimize synthetic images that closely mimic the distribution of real images, and a diversity self-distillation loss to maintain consistency in feature representation to enable the generation of synthetic images. Afterward, we use these synthetic images to fine-tune the quantization parameters with a shape-enhance data augmentation strategy. Experiment results demonstrate the superiority of the proposed framework over existing methods, with significant improvements in maintaining accuracy after quantization across various quantization configurations on the ImageNet dataset.