Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Image-based Outlier Synthesis With Training Data

Sudarshan Regmi

TL;DR

This work tackles the challenge of detecting out-of-distribution inputs under spurious correlations and fine-grained similarities without relying on external outlier data. It introduces ASCOOD, a two-stage framework that first synthesizes virtual outliers by perturbing invariant features using gradient attributions and then trains with a joint objective on standardized features to balance ID accuracy and uncertainty toward OOD. The method achieves state-of-the-art OOD detection across spurious, fine-grained, and conventional settings on seven datasets, including large-scale ImageNet-100, without external data. Overall, ASCOOD offers a data-efficient, practical pathway toward safer deployment of deep models in open-world scenarios by mitigating reliance on spurious cues and enhancing sensitivity to subtle distribution shifts.

Abstract

Out-of-distribution (OOD) detection is critical to ensure the safe deployment of deep learning models in critical applications. Deep learning models can often misidentify OOD samples as in-distribution (ID) samples. This vulnerability worsens in the presence of spurious correlation in the training set. Likewise, in fine-grained classification settings, detection of fine-grained OOD samples becomes inherently challenging due to their high similarity to ID samples. However, current research on OOD detection has focused instead largely on relatively easier (conventional) cases. Even the few recent works addressing these challenging cases rely on carefully curated or synthesized outliers, ultimately requiring external data. This motivates our central research question: ``Can we innovate OOD detection training framework for fine-grained and spurious settings \textbf{without requiring any external data at all?}" In this work, we present a unified \textbf{A}pproach to \textbf{S}purious, fine-grained, and \textbf{C}onventional \textbf{OOD D}etection (\textbf{\ASCOOD}) that eliminates the reliance on external data. First, we synthesize virtual outliers from ID data by approximating the destruction of invariant features. Specifically, we propose to add gradient attribution values to ID inputs to disrupt invariant features while amplifying true-class logit, thereby synthesizing challenging near-manifold virtual outliers. Then, we simultaneously incentivize ID classification and predictive uncertainty towards virtual outliers. For this, we further propose to leverage standardized features with z-score normalization. ASCOOD effectively mitigates impact of spurious correlations and encourages capturing fine-grained attributes. Extensive experiments across \textbf{7} datasets and and comparisons with \textbf{30+} methods demonstrate merit of ASCOOD in spurious, fine-grained and conventional settings.

Image-based Outlier Synthesis With Training Data

TL;DR

This work tackles the challenge of detecting out-of-distribution inputs under spurious correlations and fine-grained similarities without relying on external outlier data. It introduces ASCOOD, a two-stage framework that first synthesizes virtual outliers by perturbing invariant features using gradient attributions and then trains with a joint objective on standardized features to balance ID accuracy and uncertainty toward OOD. The method achieves state-of-the-art OOD detection across spurious, fine-grained, and conventional settings on seven datasets, including large-scale ImageNet-100, without external data. Overall, ASCOOD offers a data-efficient, practical pathway toward safer deployment of deep models in open-world scenarios by mitigating reliance on spurious cues and enhancing sensitivity to subtle distribution shifts.

Abstract

Out-of-distribution (OOD) detection is critical to ensure the safe deployment of deep learning models in critical applications. Deep learning models can often misidentify OOD samples as in-distribution (ID) samples. This vulnerability worsens in the presence of spurious correlation in the training set. Likewise, in fine-grained classification settings, detection of fine-grained OOD samples becomes inherently challenging due to their high similarity to ID samples. However, current research on OOD detection has focused instead largely on relatively easier (conventional) cases. Even the few recent works addressing these challenging cases rely on carefully curated or synthesized outliers, ultimately requiring external data. This motivates our central research question: ``Can we innovate OOD detection training framework for fine-grained and spurious settings \textbf{without requiring any external data at all?}" In this work, we present a unified \textbf{A}pproach to \textbf{S}purious, fine-grained, and \textbf{C}onventional \textbf{OOD D}etection (\textbf{\ASCOOD}) that eliminates the reliance on external data. First, we synthesize virtual outliers from ID data by approximating the destruction of invariant features. Specifically, we propose to add gradient attribution values to ID inputs to disrupt invariant features while amplifying true-class logit, thereby synthesizing challenging near-manifold virtual outliers. Then, we simultaneously incentivize ID classification and predictive uncertainty towards virtual outliers. For this, we further propose to leverage standardized features with z-score normalization. ASCOOD effectively mitigates impact of spurious correlations and encourages capturing fine-grained attributes. Extensive experiments across \textbf{7} datasets and and comparisons with \textbf{30+} methods demonstrate merit of ASCOOD in spurious, fine-grained and conventional settings.

Paper Structure

This paper contains 53 sections, 2 theorems, 16 equations, 21 figures, 37 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Method
  4. Image-based Outlier Synthesis
  5. Virtual Outlier Exposure (OE) Training:
  6. Experiments
  7. Results
  8. Ablation studies
  9. Related Works
  10. Conclusion
  11. Datasets
  12. Spurious setting
  13. Fine-grained setting
  14. i-ODIN: invariant-ODIN
  15. UMAP Visualization of feature space CIFAR-10 datasets and Conventional OOD (iSUN)
  16. ...and 38 more sections

Key Result

Proposition 1

The derivative of $\mathcal{L}_\text{total} = \mathcal{L}_\text{CE} + \mathcal{L}_\text{KL}$ w.r.t $k^{th}$ logit is $({\bm{p}}_{k} - {\bm{y}}_k) + \left({\bm{p}}^{\prime}_{k} - 1/C \right)$.

Figures (21)

  • Figure 1: In Waterbirds dataset Sagawa2020Distributionally, label $y \in \{\text{waterbird, landbird}\}$ is correlated with environmental feature $\mathbf{e} \in \{\text{water, land}\}$. Spurious OOD retains environmental feature $\mathbf{e}$ (water) while fine-grained OOD has its invariant feature similar to ID invariant feature $(\mathbf{x}'_\text{inv} \sim \mathbf{x}_\text{inv})$. Both present significant challenges for OOD detection.
  • Figure 2: Motivating example of the outlier synthesis pipeline. Left: An image $\mathbf{x} = \psi(\mathbf{x}_{\text{inv}}, \mathbf{e}) \in \mathcal{X}$ from the in-distribution dataset $\mathbb{D}_\text{in}$ is shown. Middle: A 2D distribution $\mathcal{G}_\text{oracle}$ is shown which signifies the presence of invariant feature in a smaller region of the image $\mathbf{x}$. Right: Corresponding outlier $\mathbf{x}^{\prime}$ is shown, which is formed by destroying the invariant feature $\mathbf{x}_\text{inv}$ of $\mathbf{x}$ through a perturbation function $\mathcal{P}_F$, having access to $\mathcal{G}_\text{oracle}$. Can we synthesize similar virtual outlier $\mathbf{x}'$ without the access of $\mathcal{G}_\text{oracle}$?
  • Figure 3: Top row: In-distribution images from the Waterbirds dataset. The first two images show waterbirds in water backgrounds, while the last two show landbirds in land backgrounds. Bottom row: Synthesized virtual outliers corresponding to the images in the top row at the latter stage of training.
  • Figure 4: Visualization of confidence scores (MSP) of (a) cross-entropy baseline and (b) ASCOOD in Waterbirds benchmark. The confidence scores of ID and (spurious and conventional (iNaturalist)) OOD are relatively well-separated in case of (b) ASCOOD in comparison to (a) cross-entropy baseline.
  • Figure 5: Comparison of $L_2$ normalization and Z-score normalization in terms of FPR@95 and AUROC in CIFAR-100 datasets.
  • ...and 16 more figures

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2