Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Zhanhui Lin, Yanlin Liu, Sanping Zhou

Abstract

Training deep neural networks with noisy labels remains a significant challenge, often leading to degraded performance. Existing methods for handling label noise typically rely on either transition matrix, noise detection, or meta-learning techniques, but they often exhibit low utilization efficiency of noisy samples and incur high computational costs. In this paper, we propose a self-training label correction framework using decoupled bilevel optimization, where a classifier and neural correction function co-evolve. Leveraging a small clean dataset, our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge, forming a closed-loop feedback system that prevents error amplification. Theoretical guarantees underpin the stability of our approach, and extensive experiments on benchmark datasets like CIFAR and Clothing1M confirm state-of-the-art performance with reduced training time, highlighting its practical applicability for learning from noisy labels.

Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels

Abstract

Training deep neural networks with noisy labels remains a significant challenge, often leading to degraded performance. Existing methods for handling label noise typically rely on either transition matrix, noise detection, or meta-learning techniques, but they often exhibit low utilization efficiency of noisy samples and incur high computational costs. In this paper, we propose a self-training label correction framework using decoupled bilevel optimization, where a classifier and neural correction function co-evolve. Leveraging a small clean dataset, our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge, forming a closed-loop feedback system that prevents error amplification. Theoretical guarantees underpin the stability of our approach, and extensive experiments on benchmark datasets like CIFAR and Clothing1M confirm state-of-the-art performance with reduced training time, highlighting its practical applicability for learning from noisy labels.
Paper Structure (43 sections, 2 theorems, 7 equations, 7 figures, 9 tables, 1 algorithm)

This paper contains 43 sections, 2 theorems, 7 equations, 7 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Strategies for Learning with Noisy Labels
  5. Self-Training Methods for LNL
  6. LNL with a Limited Clean Dataset
  7. Preliminaries
  8. Methods
  9. Noisy Posterior Simulation
  10. Robust Closed-loop Label Correction
  11. Neural Label Correction
  12. Robust Convex Combination
  13. Algorithm Details
  14. Iterative Improvement Mechanism
  15. Experiments
  16. ...and 28 more sections

Key Result

Proposition 1

Let $\hat{y}_K^* = \sum_{k=0}^K \omega_k^* \bar{y}_k$ for $K \geq 0$ (with $\bar{y}_0 \equiv \tilde{y}$), where the weight vector $\omega^* \in \Delta^K$ minimizes the expected risk $\mathcal{R}(\hat{y}_K^*)$. Then: That is, the expected risk $\mathcal{R}(\hat{y}_K^*)$ is bounded by the minimum risk of any individual component (including the original noisy label $\bar{y}_0$) and does not increase

Figures (7)

  • Figure 1: This framework provides a general overview of the approach. (a) Training the classifier on the corrected labels. (b) Feature collection and simulation of the clean-data noisy posterior. (c) Optimization of the correction function. (d) Noisy label correction.
  • Figure 2: Training dynamics of different methods on CIFAR-100 under instance-dependent noise with a level of 0.4. The vertical dashed lines represent the epochs at which the learning rate decays at 60 and 80 epochs.
  • Figure 3: Comparison of average GPU training time on CIFAR-100 (in hours) for various methods using a single RTX 4090. The values in parentheses indicate the relative ratio of training time compared to our method.
  • Figure 4: Classification accuracy on test data and clean meta data during training on Clothing1M. The vertical dashed lines represent the epochs at which the learning rate decays at 5 and 10 epoch.
  • Figure 5: Classification error on test data and clean data during training on Clothing1M. The vertical dashed lines represent the epochs at which the learning rate decays at 5 and 10 epoch.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Proposition 1
  • Proposition 1
  • proof