Sufficient Learning for Label Noise with Dual-Regularization

Abstract

DNNs inevitably over-fits to label noisy sample, resulting in poor generalization. To mitigate the influence of label noise, Co-teaching based methods tend to select potentially clean data as the training samples based on small-loss criterion. However, the small-loss criterion cannot guarantee the purity of clean samples, and blithely ignores the information involved in the noisy samples. To address this issue, we propose a simple yet effective method called SLDR. Specifically, we partition training samples into clean and noisy subsets based on the small-loss criterion to filter the errors generated in each network. Then, we employ supervised regularization to minimize intra-class distances and maximize inter-class distances in clean samples and unsupervised regularization to assess the similarity between instance features in noisy samples. Extensive experiments on mainstream benchmarks, including synthetic noisy datasets (noisy version and long-tailed noisy version), and real-world noisy datasets demonstrate superiority of the proposed method.