Combating Noisy Labels through Fostering Self- and Neighbor-Consistency

TL;DR

The paper tackles learning with noisy labels, including open-set noise, by introducing Jo-SNC, which jointly selects clean samples and regularizes model learning. It uses Jensen-Shannon divergence to assess clean-likelihood considering nearest neighbors for robust global-like selection and differentiates ID vs OOD noise via prediction divergence across two views, with per-class adaptive thresholds updated through temporal ensembling. Clean samples train conventionally with label smoothing, ID samples use partial label learning via a mean-teacher, and OOD samples leverage negative learning; a triplet consistency regularization (self-, neighbor-, and feature-consistency) further strengthens separation and representation quality. Across synthetic and real-world datasets, Jo-SNC consistently outperforms state-of-the-art baselines, showing robust performance gains and stability with ablations validating each component and hyperparameter guidance. The method offers practical impact for robust vision learning in environments with imperfect annotations and diverse noise types, with potential extensions to long-tailed data settings as future work.

Abstract

Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning. Deep networks are vulnerable to such label-corrupted samples due to the memorization effect. One major stream of previous methods concentrates on identifying clean data for training. However, these methods often neglect imbalances in label noise across different mini-batches and devote insufficient attention to out-of-distribution noisy data. To this end, we propose a noise-robust method named Jo-SNC (\textbf{Jo}int sample selection and model regularization based on \textbf{S}elf- and \textbf{N}eighbor-\textbf{C}onsistency). Specifically, we propose to employ the Jensen-Shannon divergence to measure the ``likelihood'' of a sample being clean or out-of-distribution. This process factors in the nearest neighbors of each sample to reinforce the reliability of clean sample identification. We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds. While clean samples undergo conventional training, detected in-distribution and out-of-distribution noisy samples are trained following partial label learning and negative learning, respectively. Finally, we advance the model performance further by proposing a triplet consistency regularization that promotes self-prediction consistency, neighbor-prediction consistency, and feature consistency. Extensive experiments on various benchmark datasets and comprehensive ablation studies demonstrate the effectiveness and superiority of our approach over existing state-of-the-art methods.

Figures (7)

• Figure 1: Comparative illustration of existing sample selection methods and our method. (a) Early methods typically employ a predefined ratio to drop high-loss samples within each mini-batch. (b) Recent approaches select clean samples from the entirety of training data via pre-epoch evaluation. The remaining samples are utilized in a semi-supervised manner. (c) Our method distinguish clean, ID noisy, and OOD noisy samples within each mini-batch in an (approximately) global manner, and then leverage all training data accordingly.
• Figure 2: Architecture of our proposed Jo-SNC. Firstly, a sample selection strategy (comprising clean sample identification, out-of-distribution detection, and adaptive thresholding) is introduced to divide training data into three groups (i.e., clean, ID, and OOD). Selected clean samples are leveraged for training conventionally. Conversely, identified ID and OOD noisy samples are learned by resorting to partial label learning and negative learning, respectively. Moreover, a triplet consistency regularization is established, including self-prediction consistency $\mathcal{L}_{con_s}$, neighbor-prediction consistency $\mathcal{L}_{con_n}$, and feature consistency $\mathcal{L}_{con_f}$. The final loss function is the integration of the classification loss and the consistency loss.
• Figure 3: The illustration of our sample selection strategy. The proposed sample selection relies on two JS divergence-based metrics $\mathcal{P}_{clean}$ and $\mathcal{P}_{ood}$ to determine whether a sample is clean, ID, or OOD. Moreover, we design a self-adaptive thresholding scheme to adjust selection thresholds $\tau_{clean}$ and $\tau_{ood}$ for each class in a data-driven manner.
• Figure 4: (Top) Comparison of test accuracy (%) vs. epochs with state-of-the-art sample-selection-based approaches on CIFAR80N-O datasets under various noise conditions (i.e., Sym-20%, Sym-50%, Sym-80%, and Asym-40%). (Bottom) Comparison of prediction accuracy (%) of different types of training samples (i.e., clean ones, ID noisy ones, and OOD noisy ones) on the CIFAR80N-O (Sym-20%) dataset during the training process.
• Figure 5: (Top)Comparison of (clean) sample selection performance between state-of-the-art methods and Jo-SNC on various noisy datasets.(Bottom) Comparison of (OOD noisy) sample selection performance between Jo-SRC sun2021josrc and Jo-SNC on various noisy datasets. The sample selection performance is evaluated using the F1 Score metric. These models are all trained for 200 epochs and the first 10 epochs are warmup. The evaluation of sample selection performance starts from the 11-th epoch.