AEON: Adaptive Estimation of Instance-Dependent In-Distribution and Out-of-Distribution Label Noise for Robust Learning

TL;DR

This work tackles robust image classification under realistic instance-dependent label noise that includes both in-distribution and out-of-distribution errors. It introduces AEON, a one-stage framework that jointly estimates instance-dependent noise rates $\hat{\eta}^{id}$ and $\hat{\eta}^{ood}$ using a dual-stream soft-masking mechanism, enabling end-to-end learning with supervised, unsupervised, and contrastive objectives. A novel ID+OOD instance-dependent benchmark is proposed, and AEON demonstrates state-of-the-art performance on synthetic benchmarks like CIFAR-100 and ciFAIR-100 and on real-world datasets such as Clothing1M, mini-WebVision, and WebFG-496, with competitive or superior calibration. The method offers practical benefits through efficient computation, scalable training, and improved robustness to complex noise, paving the way for automatic, instance-aware noise handling in large-scale, semi-supervised learning settings.

Abstract

Robust training with noisy labels is a critical challenge in image classification, offering the potential to reduce reliance on costly clean-label datasets. Real-world datasets often contain a mix of in-distribution (ID) and out-of-distribution (OOD) instance-dependent label noise, a challenge that is rarely addressed simultaneously by existing methods and is further compounded by the lack of comprehensive benchmarking datasets. Furthermore, even though current noisy-label learning approaches attempt to find noisy-label samples during training, these methods do not aim to estimate ID and OOD noise rates to promote their effectiveness in the selection of such noisy-label samples, and they are often represented by inefficient multi-stage learning algorithms. We propose the Adaptive Estimation of Instance-Dependent In-Distribution and Out-of-Distribution Label Noise (AEON) approach to address these research gaps. AEON is an efficient one-stage noisy-label learning methodology that dynamically estimates instance-dependent ID and OOD label noise rates to enhance robustness to complex noise settings. Additionally, we introduce a new benchmark reflecting real-world ID and OOD noise scenarios. Experiments demonstrate that AEON achieves state-of-the-art performance on both synthetic and real-world datasets

Figures (4)

• Figure 1: Different types of samples labeled as "Airplane": Clean-Set ($\bullet$) has samples with correct labels, Closed-Set ($\bullet$) contains samples with incorrect labels, where the image class ("Bird") is in the set of training labels, and Open-Set ($\bullet$) has samples with incorrect labels, where the image class ("Helicopter") is not in the set of training labels.
• Figure 2: Correlation between noise rate estimation ($\hat{\eta}$) and model performance on the ciFAIR-100 benchmark barz2020we with $40\%$ closed-set IDN ($r^{in}$) xia2020part and $40\%$ open-set noise ($r^{out}$) based on Places-IDN zhou2017places. (Left) Classification accuracy of AEON compared to PLS albert2023your over training epochs. (Right) AEON’s estimation of ID ($\hat{\eta}^{in}$) and OOD ($\hat{\eta}^{out}$) noise rates during training. For PLS, post-hoc noise estimation is shown using confidence values, as it does not directly estimate noise rates.
• Figure 3: Our proposed AEON is a novel end-to-end learning framework to simultaneously address instance-dependent closed-set and open-set label noise. The framework comprises three key components: (1) a warm-up phase establishing initial feature representations through $f_\theta(.)$; (2) a dual-stream soft masking mechanism that dynamically estimates sample reliability through energy scores (for OOD label noise detection) and loss values (for ID label noise identification), producing adaptive weights $w^{ood}$ and $w^{id}$, respectively, via noise rates estimation ($\hat{\eta}^{id}$ for closed set, and $\hat{\eta}^{ood}$ for open-set); and (3) a unified multi-objective training strategy combining supervised learning on reliable ID samples, unsupervised learning on potentially noisy ID instances, and contrastive learning for robust feature discrimination on OOD samples.
• Figure 4: Performance analysis comparing training time efficiency and classification accuracy across methods. Results on our ciFAIR-100 barz2020we benchmarks under low-noise $(r^{ood}=0.2, r^{id}=0.2)$ and high-noise $(r^{ood}=0.4, r^{id}=0.4)$ settings with instance-dependent closed-set noise xia2020part and instance-dependent Places-IDN zhou2017places open-set noise. Scatter plots illustrate the accuracy-training time trade-off, with $95\%$ confidence regions shown in gray.