Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DAVIS: OOD Detection via Dominant Activations and Variance for Increased Separation

Abid Hassan, Tuan Ngo, Saad Shafiq, Nenad Medvidovic

TL;DR

This work addresses the challenge of detecting out-of-distribution inputs by exposing and exploiting distributional cues discarded by global average pooling. The authors introduce DAVIS, a post-hoc module that enriches penultimate layer representations with channel-wise dominant activations and variance, forming a more discriminative feature vector prior to the final classifier head. DAVIS demonstrates consistent, substantial improvements in false positive rates for OOD detection across CNN architectures on CIFAR and ImageNet, and it yields synergistic gains when combined with existing baselines like Energy, ReAct, DICE, and ASH. The approach is computationally lightweight, architecture-agnostic for CNNs with GAP, robust to hyperparameter choices, and offers a principled path toward moving beyond mean-based statistics in OOD detection.

Abstract

Detecting out-of-distribution (OOD) inputs is a critical safeguard for deploying machine learning models in the real world. However, most post-hoc detection methods operate on penultimate feature representations derived from global average pooling (GAP) -- a lossy operation that discards valuable distributional statistics from activation maps prior to global average pooling. We contend that these overlooked statistics, particularly channel-wise variance and dominant (maximum) activations, are highly discriminative for OOD detection. We introduce DAVIS, a simple and broadly applicable post-hoc technique that enriches feature vectors by incorporating these crucial statistics, directly addressing the information loss from GAP. Extensive evaluations show DAVIS sets a new benchmark across diverse architectures, including ResNet, DenseNet, and EfficientNet. It achieves significant reductions in the false positive rate (FPR95), with improvements of 48.26\% on CIFAR-10 using ResNet-18, 38.13\% on CIFAR-100 using ResNet-34, and 26.83\% on ImageNet-1k benchmarks using MobileNet-v2. Our analysis reveals the underlying mechanism for this improvement, providing a principled basis for moving beyond the mean in OOD detection.

DAVIS: OOD Detection via Dominant Activations and Variance for Increased Separation

TL;DR

This work addresses the challenge of detecting out-of-distribution inputs by exposing and exploiting distributional cues discarded by global average pooling. The authors introduce DAVIS, a post-hoc module that enriches penultimate layer representations with channel-wise dominant activations and variance, forming a more discriminative feature vector prior to the final classifier head. DAVIS demonstrates consistent, substantial improvements in false positive rates for OOD detection across CNN architectures on CIFAR and ImageNet, and it yields synergistic gains when combined with existing baselines like Energy, ReAct, DICE, and ASH. The approach is computationally lightweight, architecture-agnostic for CNNs with GAP, robust to hyperparameter choices, and offers a principled path toward moving beyond mean-based statistics in OOD detection.

Abstract

Detecting out-of-distribution (OOD) inputs is a critical safeguard for deploying machine learning models in the real world. However, most post-hoc detection methods operate on penultimate feature representations derived from global average pooling (GAP) -- a lossy operation that discards valuable distributional statistics from activation maps prior to global average pooling. We contend that these overlooked statistics, particularly channel-wise variance and dominant (maximum) activations, are highly discriminative for OOD detection. We introduce DAVIS, a simple and broadly applicable post-hoc technique that enriches feature vectors by incorporating these crucial statistics, directly addressing the information loss from GAP. Extensive evaluations show DAVIS sets a new benchmark across diverse architectures, including ResNet, DenseNet, and EfficientNet. It achieves significant reductions in the false positive rate (FPR95), with improvements of 48.26\% on CIFAR-10 using ResNet-18, 38.13\% on CIFAR-100 using ResNet-34, and 26.83\% on ImageNet-1k benchmarks using MobileNet-v2. Our analysis reveals the underlying mechanism for this improvement, providing a principled basis for moving beyond the mean in OOD detection.
Paper Structure (32 sections, 2 theorems, 32 equations, 8 figures, 25 tables)

This paper contains 32 sections, 2 theorems, 32 equations, 8 figures, 25 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Dominant Activations and Variance for Increased Separation
  5. OOD Detection with DAVIS
  6. Empirical Evaluation of DAVIS
  7. CIFAR Evaluation
  8. ImageNet Evaluation
  9. Comparison with Other Baselines
  10. Near-OOD Evaluation
  11. Hyperparameter Selection
  12. Discussion
  13. Ablation Studies
  14. Conclusion
  15. Description of Baseline Methods
  16. ...and 17 more sections

Key Result

Lemma 1

Given Observation obs:obs1, the separation gap of the combined mean-and-standard-deviation feature is greater than or equal to that of the mean feature alone:

Figures (8)

  • Figure 1: Dominant activations provide a stronger OOD signal than mean activations. The plot shows the average activation gap between ID (CIFAR-10) and OOD (Texture) samples for each unit in the penultimate layer of a pre-trained ResNet-18. The gap derived from the dominant (maximum) activation (blue) is consistently and significantly larger than the gap from the standard mean activation (orange).
  • Figure 2: Using dominant activations improves OOD score separation. Left: OOD scores based on standard mean activations show significant overlap between the ID (CIFAR-10) and OOD (Texture) distributions, leading to poor separability. Right: Leveraging dominant (maximum) activations shifts the OOD score distribution away from the ID scores. Both plots show energy scores from a ResNet-18.
  • Figure 3: Feature statistics for ID (ImageNet) vs. OOD (Texture) samples on an efficientNet-b0 backbone. While mean $\mu(\mathbf{x})$ show poor separation, both the standard deviation $\sigma(\mathbf{x})$ and maximum $m(\mathbf{x})$ statistics maintain a clear separation between ID and OOD activations.
  • Figure 4: Unit-wise comparison of statistical features for ID vs. OOD samples, with values averaged over the entire test set. Across a majority of feature dimensions, the mean ($\mu(\mathbf{x})$), standard deviation ($\sigma(\mathbf{x})$), and maximum ($m(\mathbf{x})$) statistics all exhibit consistently higher values for ID samples (blue) than for OOD samples (orange). Results are shown for a ResNet-50 model with ImageNet-1K as the ID dataset and Texture as the OOD dataset. This trend holds consistently across other architectures and data combinations.
  • Figure 5: Comparison of the separation gap $\Delta$ achieved by different statistical features, averaged over all test samples. Left: It demonstrate that incorporating the standard deviation $\Delta_{\mu,\sigma}$ yields a larger separation gap than using the mean activation alone $\Delta_\mu$. Right: It demonstrate that using the maximum activation $\Delta_m$ yields a larger separation gap than using the mean activation $\Delta_\mu$. Results are shown for a ResNet-50 model with ImageNet as the ID dataset and Texture as the OOD dataset. This finding holds consistently across other architectures and data combinations.
  • ...and 3 more figures

Theorems & Definitions (5)

  • Definition 1
  • Lemma 1
  • proof
  • Theorem 1
  • proof