Set a Thief to Catch a Thief: Combating Label Noise through Noisy Meta Learning

TL;DR

This paper addresses learning from noisy labels by removing the need for extra clean validation data. It introduces STCT, a noisy meta label correction framework that uses a randomly sampled noisy validation set drawn from the training data to guide label correction via an alternating between noisy meta correction and semi-supervised representation learning. The approach is theoretically motivated by consistency results between optimal classifiers on noisy and clean distributions and is operationalized through a non-nested optimization that decouples representation learning from label correction. Extensive experiments on CIFAR-10/100 under symmetric and asymmetric noise, as well as real-world datasets like CIFAR-10N, CIFAR-100N, and Clothing-1M, demonstrate superior label correction and classification performance, particularly at high noise rates. STCT thus offers a practical, annotation-light alternative to traditional meta-learning label correction methods with strong empirical upside and potential broad impact for robust learning in noisy-label regimes.

Abstract

Learning from noisy labels (LNL) aims to train high-performance deep models using noisy datasets. Meta learning based label correction methods have demonstrated remarkable performance in LNL by designing various meta label rectification tasks. However, extra clean validation set is a prerequisite for these methods to perform label correction, requiring extra labor and greatly limiting their practicality. To tackle this issue, we propose a novel noisy meta label correction framework STCT, which counterintuitively uses noisy data to correct label noise, borrowing the spirit in the saying ``Set a Thief to Catch a Thief''. The core idea of STCT is to leverage noisy data which is i.i.d. with the training data as a validation set to evaluate model performance and perform label correction in a meta learning framework, eliminating the need for extra clean data. By decoupling the complex bi-level optimization in meta learning into representation learning and label correction, STCT is solved through an alternating training strategy between noisy meta correction and semi-supervised representation learning. Extensive experiments on synthetic and real-world datasets demonstrate the outstanding performance of STCT, particularly in high noise rate scenarios. STCT achieves 96.9% label correction and 95.2% classification performance on CIFAR-10 with 80% symmetric noise, significantly surpassing the current state-of-the-art.

Figures (10)

• Figure 1: Comparison between (a) traditional meta learning based label correction framework and (b) STCT. STCT utilizes noisy validation sets for model performance evaluation, allowing label correction without using any extra clean data compared to traditional meta-learning methods.
• Figure 2: Comparison with other label correction methods on CIFAR-10 with symmetric noise. STCT's label correction capability significantly surpasses the competing methods.
• Figure 3: The overall framework of STCT. NMC constructs a linear model in the embedding space and uses the sampled noisy validation set to evaluate model performance, enabling label correction. SRL selects clean samples from the corrected training labels and uses semi-supervised learning to improve the inter-class separability of the embedding features extracted by the encoder.
• Figure 4: The training process of STCT on CIFAR-10 with 80% symmetric noise. (a) The testing accuracy and training loss curves of SRL. (b) The corrected training label accuracy curve of NMC.
• Figure 5: Visualization of the distribution of different generated validation sets and clean data in the embedding space under 50% symmetric label noise. The blue region represents the clean distribution, and the red region represents the distribution estimated by the generated validation sets. The closer the two regions are, the more effective the validation set is. (a) Loss-based method; (b) KNN-based method; (c) Clustering-based method.