ProAug: Prototype-Based Augmentation for Long-Tailed Image Classification

Abstract

Real-world data often exhibit long-tailed distributions with heavy class imbalance, which deteriorates the generalization performance of the classifier. To mitigate this problem, we propose a novel Prototype-based Augmentation framework (ProAug) to address the data scarcity issue by augmenting the feature space for tail classes. Our ProAug consists of a prototype construction branch and a dynamic augmentation branch. The prototype-based dictionary is optimized with category-aware margin loss to learn multi-center and discriminative prototypes for each category. In the dynamic augmentation branch, we aim to produce high-quality tail-class features by dynamically composing context-similar prototypes with an attention mechanism. Moreover, to further improve the reliability of prototypes and the quality of augmented features, a meta-update strategy is adopted to calibrate two branches of ProAug to boost performance. Extensive empirical results on CIFAR-LT-10/100, ImageNet-LT, and iNaturalist 2018 demonstrate the effectiveness of our method.