Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection

Haoyue Bai, Gregory Canal, Xuefeng Du, Jeongyeol Kwon, Robert Nowak, Yixuan Li

TL;DR

This work tackles the challenge of deploying classifiers under both covariate and semantic distribution shifts by introducing Scone, a margin-based framework that uses unlabeled wild data. Central to Scone is an energy function with a negative margin constraint that forces in-distribution samples to lie far from the OOD boundary, enabling improved OOD generalization to covariate shifts while maintaining robust OOD detection for semantic shifts. The authors provide theoretical insights linking the margin to covariate-shift generalization and demonstrate strong empirical gains across CIFAR-10/10-C, SVHN, and ImageNet-100 settings, outperforming baselines specialized in either task. The approach offers a practical, data-efficient path for robust open-world learning, with publicly available code and broad applicability to real-world deployment.

Abstract

Modern machine learning models deployed in the wild can encounter both covariate and semantic shifts, giving rise to the problems of out-of-distribution (OOD) generalization and OOD detection respectively. While both problems have received significant research attention lately, they have been pursued independently. This may not be surprising, since the two tasks have seemingly conflicting goals. This paper provides a new unified approach that is capable of simultaneously generalizing to covariate shifts while robustly detecting semantic shifts. We propose a margin-based learning framework that exploits freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. We show both empirically and theoretically that the proposed margin constraint is the key to achieving both OOD generalization and detection. Extensive experiments show the superiority of our framework, outperforming competitive baselines that specialize in either OOD generalization or OOD detection. Code is publicly available at https://github.com/deeplearning-wisc/scone.

Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection

TL;DR

This work tackles the challenge of deploying classifiers under both covariate and semantic distribution shifts by introducing Scone, a margin-based framework that uses unlabeled wild data. Central to Scone is an energy function with a negative margin constraint that forces in-distribution samples to lie far from the OOD boundary, enabling improved OOD generalization to covariate shifts while maintaining robust OOD detection for semantic shifts. The authors provide theoretical insights linking the margin to covariate-shift generalization and demonstrate strong empirical gains across CIFAR-10/10-C, SVHN, and ImageNet-100 settings, outperforming baselines specialized in either task. The approach offers a practical, data-efficient path for robust open-world learning, with publicly available code and broad applicability to real-world deployment.

Abstract

Modern machine learning models deployed in the wild can encounter both covariate and semantic shifts, giving rise to the problems of out-of-distribution (OOD) generalization and OOD detection respectively. While both problems have received significant research attention lately, they have been pursued independently. This may not be surprising, since the two tasks have seemingly conflicting goals. This paper provides a new unified approach that is capable of simultaneously generalizing to covariate shifts while robustly detecting semantic shifts. We propose a margin-based learning framework that exploits freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. We show both empirically and theoretically that the proposed margin constraint is the key to achieving both OOD generalization and detection. Extensive experiments show the superiority of our framework, outperforming competitive baselines that specialize in either OOD generalization or OOD detection. Code is publicly available at https://github.com/deeplearning-wisc/scone.
Paper Structure (34 sections, 1 theorem, 10 equations, 3 figures, 10 tables)

This paper contains 34 sections, 1 theorem, 10 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Labeled in-distribution data.
  4. Unlabeled wild data.
  5. Methodology
  6. Preliminaries
  7. Proposed Method
  8. Motivation.
  9. Theoretical Insights: Relationship Between OOD Generalization and Energy Margin
  10. Enforcing Margin in Practice
  11. Experiments
  12. Experimental Setup
  13. Datasets and evaluation metrics.
  14. Results and Discussion
  15. Effect of different mixing ratios.
  16. ...and 19 more sections

Key Result

Proposition 3.1

(Informal) Under some assumptions, if $\eta < - \log2 - \frac{1}{2} L \delta$ then each covariate-shifted point is classified correctly and is detected as semantic in.

Figures (3)

  • Figure 1: Illustration of three types of data that can organically arise when deploying models in the open world: (1) in-distribution (ID) data (e.g., car on a sunny day), (2) covariate-shifted OOD data (e.g., car in the snow), and (3) semantic-shifted OOD data (e.g., a deer). Our framework enables leveraging the wild mixture (containing all three types of data) for OOD generalization and OOD detection.
  • Figure 2: Illustration of the impact of energy margin $\eta$ on the placement of the OOD detection boundary.
  • Figure 3: (a)-(b) Energy score distributions for WOODS vs. our method. Different colors represent the different types of test data: CIFAR-10 as $\mathbb{P}_{\text{in}}$ (blue), CIFAR-10-C as $\mathbb{P}_\text{out}^\text{covariate}$ (green), and SVHN as $\mathbb{P}_\text{out}^\text{semantic}$ (gray). (c)-(d): T-SNE visualization of the image embeddings using WOODS vs. our method.

Theorems & Definitions (1)

  • Proposition 3.1