Active Learning for Long-Tailed Annotation

Abstract

Active learning (AL) is an effective method to balance annotation costs and model performance under resource-constrained circumstances. Most existing AL studies are typically designed for class-balanced datasets. However, the ubiquity of long-tailed distributions in real-world scenarios largely restricts the applicability of those AL methods. To tackle this problem, we propose a new active learning framework, namely long-tailed active learning (LTAL). The LTAL framework divides the long-tailed dataset into constantly evolving in-distribution (ID) and out-of-distribution (OOD) samples, and views the tail samples as OOD samples distinct from the head ones, thus intuitively converting the LTA problem into an iterative OOD detection task. We leverage an energy-based OOD detection approach with a well-designed class-imbalanced energy regularization loss to further extend the energy gap between head and tail classes, encouraging the model to select more unlabeled tail samples with higher free energy values. Experimental results show that despite its conceptual simplicity, the proposed method significantly outperforms competitive baselines.