Active Negative Loss: A Robust Framework for Learning with Noisy Labels

TL;DR

This work introduces Active Negative Loss (ANL), a robust learning framework for noisy labels that substitutes Normalized Negative Loss Functions (NNLFs) for MAE within the Active Passive Loss (APL) setup. NNLFs, built from complementary-label learning, vertical flipping, and normalization, transform active losses into robust passive losses and, when combined with normalized active losses, yield ANLLoss with strong noise tolerance. The authors prove symmetry and noise-tolerance properties for NNLFs and ANL, provide gradient insights, and validate the approach across CIFAR, WebVision, Animal-10N, Clothing-1M, and ISIC-2017 segmentation, showing state-of-the-art or competitive performance under various label-noise regimes. An entropy-based regularization further mitigates label-imbalance effects in non-symmetric noise, and the method demonstrates scalability to large real-world datasets with practical code availability.

Abstract

Deep supervised learning has achieved remarkable success across a wide range of tasks, yet it remains susceptible to overfitting when confronted with noisy labels. To address this issue, noise-robust loss functions offer an effective solution for enhancing learning in the presence of label noise. In this work, we systematically investigate the limitation of the recently proposed Active Passive Loss (APL), which employs Mean Absolute Error (MAE) as its passive loss function. Despite the robustness brought by MAE, one of its key drawbacks is that it pays equal attention to clean and noisy samples; this feature slows down convergence and potentially makes training difficult, particularly in large-scale datasets. To overcome these challenges, we introduce a novel loss function class, termed Normalized Negative Loss Functions (NNLFs), which serve as passive loss functions within the APL framework. NNLFs effectively address the limitations of MAE by concentrating more on memorized clean samples. By replacing MAE in APL with our proposed NNLFs, we enhance APL and present a new framework called Active Negative Loss (ANL). Moreover, in non-symmetric noise scenarios, we propose an entropy-based regularization technique to mitigate the vulnerability to the label imbalance. Extensive experiments demonstrate that the new loss functions adopted by our ANL framework can achieve better or comparable performance to state-of-the-art methods across various label noise types and in image segmentation tasks. The source code is available at: https://github.com/Virusdoll/Active-Negative-Loss.

Figures (8)

• Figure 1: "Vertical flipping" operation. $\mathcal{L}(f(\boldsymbol{x}),k)$ is an active loss function, e.g., CE. Loss function $A - \mathcal{L}(f(\boldsymbol{x}),k)$ is obtained by flipping $\mathcal{L}(f(\boldsymbol{x}),k)$ vertically with axis $\text{loss}=\frac{1}{2}A$.
• Figure 2: Marginal probability of each class on the CIFAR-10 training set with 40% asymmetric label noise, along with model predictions using different approaches. ANL-CE/ANL-CE* is introduced in \ref{['eq:anl-ce']}/\ref{['eq:anl-ce-star']}, respectively.
• Figure 3: Training and test accuracies of different loss functions. (a) - (e): CIFAR-10 under 0.8 symmetric noise. (f) - (j): CIFAR-100 under 0.6 symmetric noise. The accuracies of noisy samples in training set should be as low as possible, since they are mislabeled.
• Figure 4: Example segmentation results using ANL-CE on ISIC-2017 dataset under the noise scenario with $\eta_\alpha = 0.7$ and $\eta_\beta = 0.7$. The images show predicted masks under three noise conditions: affine masks, dilated masks, and eroded masks. Each row represents, from top to bottom: original images, ground truth masks, noisy masks, and our predicted masks.
• Figure 5: Test accuracies of ANL-CE on CIFAR-10 under 0.8 symmetric noise with different regularization methods and different parameters. $\delta$ is the weight of regularization term of ANL-CE.

Theorems & Definitions (22)

• Definition 1: Active loss function
• Definition 2: Passive loss function
• Definition 3: Symmetric loss function
• Theorem 1
• Definition 4: Noise tolerance ghosh2017robust.
• Theorem 2
• Theorem 3
• Theorem 4
• Theorem 5
• Theorem 6