Robust Classification with Noisy Labels Based on Posterior Maximization

TL;DR

The paper develops f-divergence based Posterior Maximization Learning (f-PML) for classification with noisy labels, showing how to achieve robustness via two routes: (i) correcting the objective during training so the learned network matches the clean-data network, and (ii) correcting the posterior estimator at test time to recover clean MAP decisions. It proves that f-PML is robust to symmetric label noise for any f-divergence and that cross-entropy, within this framework, inherits this robustness; it also provides convergence analyses and highlights the relationship to active-passive losses. The authors validate the approach on binary and multi-class tasks, demonstrating competitive performance against a broad set of baselines and state-of-the-art noise-robust methods, especially when combined with refined training strategies. The work offers practical guidance for deploying robust classifiers under label noise and clarifies misconceptions about the robustness of CE in symmetric-noise settings.

Abstract

Designing objective functions robust to label noise is crucial for real-world classification algorithms. In this paper, we investigate the robustness to label noise of an $f$-divergence-based class of objective functions recently proposed for supervised classification, herein referred to as $f$-PML. We show that, in the presence of label noise, any of the $f$-PML objective functions can be corrected to obtain a neural network that is equal to the one learned with the clean dataset. Additionally, we propose an alternative and novel correction approach that, during the test phase, refines the posterior estimated by the neural network trained in the presence of label noise. Then, we demonstrate that, even if the considered $f$-PML objective functions are not symmetric, they are robust to symmetric label noise for any choice of $f$-divergence, without the need for any correction approach. This allows us to prove that the cross-entropy, which belongs to the $f$-PML class, is robust to symmetric label noise. Finally, we show that such a class of objective functions can be used together with refined training strategies, achieving competitive performance against state-of-the-art techniques of classification with label noise.

Figures (1)

• Figure 1: Proposed framework in the absence (left) and presence (right) of label noise. To achieve robustness with label noise, the objective function correction (green) is performed during training to obtain the clean estimate of the posterior as the output of the neural network. Alternatively, the posterior correction (blue) is implemented during test by correcting the posterior estimate. In the case of symmetric noise, the proposed framework does not require any correction technique. The dashed arrows indicate the model update through backpropagation.

Theorems & Definitions (18)

• Theorem 3.1
• Corollary 3.2
• Theorem 3.3
• Theorem 3.4
• Corollary 3.5
• Theorem 3.6
• Theorem 3.7
• Theorem 3.8
• Theorem 3.9
• proof