Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning with Noisy Labels through Learnable Weighting and Centroid Similarity

Farooq Ahmad Wani, Maria Sofia Bucarelli, Fabrizio Silvestri

Abstract

We introduce a novel method for training machine learning models in the presence of noisy labels, which are prevalent in domains such as medical diagnosis and autonomous driving and have the potential to degrade a model's generalization performance. Inspired by established literature that highlights how deep learning models are prone to overfitting to noisy samples in the later epochs of training, we propose a strategic approach. This strategy leverages the distance to class centroids in the latent space and incorporates a discounting mechanism, aiming to diminish the influence of samples that lie distant from all class centroids. By doing so, we effectively counteract the adverse effects of noisy labels. The foundational premise of our approach is the assumption that samples situated further from their respective class centroid in the initial stages of training are more likely to be associated with noise. Our methodology is grounded in robust theoretical principles and has been validated empirically through extensive experiments on several benchmark datasets. Our results show that our method consistently outperforms the existing state-of-the-art techniques, achieving significant improvements in classification accuracy in the presence of noisy labels. The code for our proposed loss function and supplementary materials is available at https://github.com/wanifarooq/NCOD

Learning with Noisy Labels through Learnable Weighting and Centroid Similarity

Abstract

We introduce a novel method for training machine learning models in the presence of noisy labels, which are prevalent in domains such as medical diagnosis and autonomous driving and have the potential to degrade a model's generalization performance. Inspired by established literature that highlights how deep learning models are prone to overfitting to noisy samples in the later epochs of training, we propose a strategic approach. This strategy leverages the distance to class centroids in the latent space and incorporates a discounting mechanism, aiming to diminish the influence of samples that lie distant from all class centroids. By doing so, we effectively counteract the adverse effects of noisy labels. The foundational premise of our approach is the assumption that samples situated further from their respective class centroid in the initial stages of training are more likely to be associated with noise. Our methodology is grounded in robust theoretical principles and has been validated empirically through extensive experiments on several benchmark datasets. Our results show that our method consistently outperforms the existing state-of-the-art techniques, achieving significant improvements in classification accuracy in the presence of noisy labels. The code for our proposed loss function and supplementary materials is available at https://github.com/wanifarooq/NCOD
Paper Structure (25 sections, 2 theorems, 11 equations, 19 figures, 4 tables)

This paper contains 25 sections, 2 theorems, 11 equations, 19 figures, 4 tables.

Table of Contents

  1. Keywords
  2. Introduction
  3. Related Work
  4. Evolution of Training
  5. Latent Space Representation
  6. Distribution around Class Cluster Centers
  7. Preliminaries
  8. Methodology
  9. Class Embeddings
  10. Outlier Discounting
  11. Cross-Dataset Noise Analysis of Loss Functions
  12. Experiments
  13. Training details for NCOD and NCOD+
  14. Definition of label Noise
  15. NCOD+
  16. ...and 10 more sections

Key Result

Theorem 5.1

Let $(x_i, y_i)$ be a sample, $\theta^t$ be the parameters of the network at epoch $t$, and $u^t_i$ be the parameter for outlier discounting relative to sample $i$ at epoch $t$. Let $\hat{c}^t_i$ be the prediction of the network at time $t$, and $y_i$ be the class of sample $i$. Suppose $\hat{c}^t_i

Figures (19)

  • Figure 1: Sample embeddings of four classes from CIFAR-100 with 20% symmetrical noise. Colors represent classes, and shapes distinguish noisy and pure samples: Blue (square: pure, pentagon: noisy), Red (circle: pure, diamond: noisy), Green (triangle-up: pure, star: noisy), and Purple (triangle-down: pure, hexagon: noisy).
  • Figure 2: Distribution of four classes from CIFAR-100 with 20% symmetrical noise. ep. is an abbreviation for "epoch".
  • Figure 3: Distribution of four classes for noisy and clean labels from CIFAR-100 with 20% symmetrical noise using CE Loss. ep. is an abbreviation for "epoch".
  • Figure 4: Distribution of four classes of noisy and clean labels from CIFAR-100 with 20% symmetrical noise using NCOD Loss. ep. is an abbreviation for "epoch".
  • Figure 5: CIFAR 100 $50\%$ symmetrical noise.
  • ...and 14 more figures

Theorems & Definitions (7)

  • Theorem 5.1
  • proof
  • Remark 5.2
  • proof
  • Theorem : \ref{['increasing_of_u']}
  • proof
  • proof