An Efficient Sample Utilization Method for Deep Learning Based on Class Uncertainty

Abstract

Deep learning has achieved success across many domains when sufficient training samples are available. However, the commonly used mini-batch stochastic gradient descent (SGD) training paradigm treats each sample equally, resulting in massive computational waste on samples that are easily identifiable. In contrast, low-quality samples, such as those with erroneous labels, can negatively impact the training process. To address these issues, we propose a training sample utilization method based on sample uncertainty. Once the model has acquired preliminary decision-making abilities, the class uncertainty for each sample can be evaluated within a training epoch. Subsequently, the samples are probabilistically selected based on their uncertainty for the next epoch. Experiments conducted on the GSC v2 and CIFAR-10 datasets demonstrate that the proposed method can reduce training time by over 32% and 58%, respectively, with only a loss of 1% performance. Additionally, the method has the capability to mitigate the adverse effects of samples with erroneous labels.