Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LoD: Loss-difference OOD Detection by Intentionally Label-Noisifying Unlabeled Wild Data

Chuanxing Geng, Qifei Li, Xinrui Wang, Dong Liang, Songcan Chen, Pong C. Yuen

TL;DR

A novel loss-difference OOD detection framework (LoD) is proposed by intentionally label-noisifying unlabeled wild data, enabling labeled ID data and OOD data in unlabeled wild data to jointly dominate the models' learning but also ensuring the distinguishability of the losses between ID and OOD samples in unlabeled wild data.

Abstract

Using unlabeled wild data containing both in-distribution (ID) and out-of-distribution (OOD) data to improve the safety and reliability of models has recently received increasing attention. Existing methods either design customized losses for labeled ID and unlabeled wild data then perform joint optimization, or first filter out OOD data from the latter then learn an OOD detector. While achieving varying degrees of success, two potential issues remain: (i) Labeled ID data typically dominates the learning of models, inevitably making models tend to fit OOD data as IDs; (ii) The selection of thresholds for identifying OOD data in unlabeled wild data usually faces dilemma due to the unavailability of pure OOD samples. To address these issues, we propose a novel loss-difference OOD detection framework (LoD) by \textit{intentionally label-noisifying} unlabeled wild data. Such operations not only enable labeled ID data and OOD data in unlabeled wild data to jointly dominate the models' learning but also ensure the distinguishability of the losses between ID and OOD samples in unlabeled wild data, allowing the classic clustering technique (e.g., K-means) to filter these OOD samples without requiring thresholds any longer. We also provide theoretical foundation for LoD's viability, and extensive experiments verify its superiority.

LoD: Loss-difference OOD Detection by Intentionally Label-Noisifying Unlabeled Wild Data

TL;DR

A novel loss-difference OOD detection framework (LoD) is proposed by intentionally label-noisifying unlabeled wild data, enabling labeled ID data and OOD data in unlabeled wild data to jointly dominate the models' learning but also ensuring the distinguishability of the losses between ID and OOD samples in unlabeled wild data.

Abstract

Using unlabeled wild data containing both in-distribution (ID) and out-of-distribution (OOD) data to improve the safety and reliability of models has recently received increasing attention. Existing methods either design customized losses for labeled ID and unlabeled wild data then perform joint optimization, or first filter out OOD data from the latter then learn an OOD detector. While achieving varying degrees of success, two potential issues remain: (i) Labeled ID data typically dominates the learning of models, inevitably making models tend to fit OOD data as IDs; (ii) The selection of thresholds for identifying OOD data in unlabeled wild data usually faces dilemma due to the unavailability of pure OOD samples. To address these issues, we propose a novel loss-difference OOD detection framework (LoD) by \textit{intentionally label-noisifying} unlabeled wild data. Such operations not only enable labeled ID data and OOD data in unlabeled wild data to jointly dominate the models' learning but also ensure the distinguishability of the losses between ID and OOD samples in unlabeled wild data, allowing the classic clustering technique (e.g., K-means) to filter these OOD samples without requiring thresholds any longer. We also provide theoretical foundation for LoD's viability, and extensive experiments verify its superiority.
Paper Structure (36 sections, 4 theorems, 24 equations, 5 figures, 9 tables, 1 algorithm)

This paper contains 36 sections, 4 theorems, 24 equations, 5 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Out-of-Distribution Detection
  4. Training Neural Networks with Label Noises
  5. Methodology
  6. Problem Formulation
  7. Labeled ID Data
  8. Unlabeled Wild Data
  9. Learning Goal
  10. Loss-Difference OOD Detection Framework
  11. Loss-difference OOD Filtering Module
  12. Remark.
  13. OOD Detector Learning Module
  14. Theoretical Analysis
  15. Mitigation of The Model Bias
  16. ...and 21 more sections

Key Result

Lemma 1

Denote by {$\bm{\theta}_t$} the iterates of gradient descent with step size $\eta$. For any $\Delta\in(0,1/2)$, there exists a constant $\delta_{\Delta}$, depending only on $\Delta$, such that if $\delta\leq \delta_{\Delta}$, then with high probability $1-o(1)$, there exists a $T=\Omega(1/\eta)$ suc

Figures (5)

  • Figure 1: The cross-entropy loss changes of ID (label-noise) and OOD (label-clean) samples in unlabeled wild data when they are intentionally labeled as $K+1$-th class. These two types of samples typically exhibit different loss curves due to the differences in how learning progresses for each.
  • Figure 2: Overview of the loss-difference OOD detection framework by intentionally label-noisifying unlabeled wild data.
  • Figure 3: The mean cross-entropy loss curves respectively for all ID (label-noise) and OOD samples (label-clean) in unlabeled wild data when they are intentionally labeled as $K+1$-th class.
  • Figure 4: Experiments in different rations ($|\mathcal{B}_{\text{in}}^{\text{train}}|/|\mathcal{B}_{\text{wild}}|$) on standard benchmarks (dashed lines) and hard benchmarks (solid lines).
  • Figure 5: The impacts of training epochs on results respectively in standard and hard benchmarks.

Theorems & Definitions (5)

  • Lemma 1: Early-learning succeeds liu2020early
  • Proposition 1
  • Lemma 1: Early-learning succeeds
  • Proposition 1
  • proof