Data-Free Post-Training Quantization with Block-wise Enhanced Sample Generation

Abstract

Data-free quantization is known for quantizing a pre-trained deep neural network without access to any training data, which applies to many real-world scenarios in that the training data is unavailable due to security, user privacy, or proprietary concerns. Most of the existing data-free quantization methods adopt a generator-quantization framework, which generator network to synthesize fake samples and Quantization-Aware Training (QAT) to quantize model. While the combination of the generator network and QAT can result in good accuracy for quantized models, the diversity of generated samples is lacking and quantizing a single model may take over 10 hours, which contrasts with Post-Training Quantization (PTQ)’s time-saving potential but poor accuracy. In order to address these issues, we have made improvements to the data generation and quantization process. In detail, 1) We propose Generator Exploration Enhancement (GEE) for utilizing the batch normalization statistics and adversarial sample exploration to enhance the quality and diversity of synthetic samples; 2) We introduce Block-wise Sample Generation (BSG) to collectively optimize individual blocks and the generator, leveraging PTQ as a foundation to boost workflow efficiency. Experiment results show that our proposed method improves both the performance and the efficiency of the data-free quantization compared to that of existing methods. Significantly, BSG achieves an 18% accuracy improvement and reduces quantization time by over 50% for 3-bit ResNet-18 in ImageNet tasks, surpassing the current state-of-the-art QAT method.