Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection

TL;DR

DynaCor is a framework that distinguishes incorrectly labeled instances from correctly labeled ones based on the dynamics of the training signals based on the dynamics of the training signals and shows strong robustness to various noise types and noise rates.

Abstract

Label noise, commonly found in real-world datasets, has a detrimental impact on a model's generalization. To effectively detect incorrectly labeled instances, previous works have mostly relied on distinguishable training signals, such as training loss, as indicators to differentiate between clean and noisy labels. However, they have limitations in that the training signals incompletely reveal the model's behavior and are not effectively generalized to various noise types, resulting in limited detection accuracy. In this paper, we propose DynaCor framework that distinguishes incorrectly labeled instances from correctly labeled ones based on the dynamics of the training signals. To cope with the absence of supervision for clean and noisy labels, DynaCor first introduces a label corruption strategy that augments the original dataset with intentionally corrupted labels, enabling indirect simulation of the model's behavior on noisy labels. Then, DynaCor learns to identify clean and noisy instances by inducing two clearly distinguishable clusters from the latent representations of training dynamics. Our comprehensive experiments show that DynaCor outperforms the state-of-the-art competitors and shows strong robustness to various noise types and noise rates.

Figures (6)

• Figure 1: The proposed DynaCor framework consists of three steps: (1) Corrupted dataset construction generates the augmented images with corrupted labels, likely resulting in noisy labels, in order to provide guidance for discrimination between clean and noisy labels. (2) Training dynamics generation collects the trajectory of training signals for both the original and corrupted datasets by training a classifier. (3) Noisy label detection is performed by discovering two distinguishable clusters of dynamics representations, and for this, the dynamics encoder is optimized to enhance both cluster cohesion and alignment between the original and the corrupted datasets.
• Figure 2: F1 score (%) changes with respect to corruption rate $(\gamma)$ on CIFAR10 in supervised and unsupervised settings using CLIP w/ MLP (Left) and ResNet34 (Right) as classifiers.
• Figure 3: Compatibility analysis of Dividemix with DynaCor on CIFAR100 over "Asym." and "Inst." with respect to noise rate
• Figure 4: Dataset construction for supervised learning.
• Figure 5: Comparison of detection F1 score (%) achieved by the binary classifiers trained using the training dynamics (comb-pattern bar and star marker in legend) versus those trained with the summarized one for various noise types on CIFAR-100. Prob. and Logit diff. indicate the types of training signals in Table \ref{['tbl:train_signal']}. Noise rates of Sym., Asym., and Instance are 0.6, 0.4, and 0.3, respectively. The human-induced noise has noise rates of 0.4. CLIP w/ MLP (Left) and Resnet34 (Right) are used for training dynamics generation.

Theorems & Definitions (2)

• Proposition 1: Lower bound of $\eta_\gamma$
• Proposition 2: Lower bound of $\eta_\gamma$