Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Catalyst: Out-of-Distribution Detection via Elastic Scaling

Abid Hassan, Tuan Ngo, Saad Shafiq, Nenad Medvidovic

TL;DR

Catalyst addresses the limitations of relying solely on penultimate GAP features for OOD detection by exploiting raw channel-wise statistics from the pre-pooling activation maps. It computes an input-dependent scaling γ from per-channel statistics (mean, std, max) with clipping, and elastically scales a baseline score, primarily via multiplicative fusion, to enlarge the ID–OOD separation. The approach is plug-and-play, improving a wide range of baselines (Energy, ReAct, DICE, ASH, SCALE, and KNN) without retraining and with negligible computational overhead. Across CIFAR and ImageNet, Catalyst demonstrates substantial, consistent improvements in FPR95 and AUROC, and ablations validate the design choices, including layer choice and the fusion strategy. The work highlights the untapped value of pre-pooling statistics for robust OOD detection and outlines avenues for extending the framework to transformer-based architectures.

Abstract

Out-of-distribution (OOD) detection is critical for the safe deployment of deep neural networks. State-of-the-art post-hoc methods typically derive OOD scores from the output logits or penultimate feature vector obtained via global average pooling (GAP). We contend that this exclusive reliance on the logit or feature vector discards a rich, complementary signal: the raw channel-wise statistics of the pre-pooling feature map lost in GAP. In this paper, we introduce Catalyst, a post-hoc framework that exploits these under-explored signals. Catalyst computes an input-dependent scaling factor ($γ$) on-the-fly from these raw statistics (e.g., mean, standard deviation, and maximum activation). This $γ$ is then fused with the existing baseline score, multiplicatively modulating it -- an ``elastic scaling'' -- to push the ID and OOD distributions further apart. We demonstrate Catalyst is a generalizable framework: it seamlessly integrates with logit-based methods (e.g., Energy, ReAct, SCALE) and also provides a significant boost to distance-based detectors like KNN. As a result, Catalyst achieves substantial and consistent performance gains, reducing the average False Positive Rate by 32.87 on CIFAR-10 (ResNet-18), 27.94% on CIFAR-100 (ResNet-18), and 22.25% on ImageNet (ResNet-50). Our results highlight the untapped potential of pre-pooling statistics and demonstrate that Catalyst is complementary to existing OOD detection approaches.

Catalyst: Out-of-Distribution Detection via Elastic Scaling

TL;DR

Catalyst addresses the limitations of relying solely on penultimate GAP features for OOD detection by exploiting raw channel-wise statistics from the pre-pooling activation maps. It computes an input-dependent scaling γ from per-channel statistics (mean, std, max) with clipping, and elastically scales a baseline score, primarily via multiplicative fusion, to enlarge the ID–OOD separation. The approach is plug-and-play, improving a wide range of baselines (Energy, ReAct, DICE, ASH, SCALE, and KNN) without retraining and with negligible computational overhead. Across CIFAR and ImageNet, Catalyst demonstrates substantial, consistent improvements in FPR95 and AUROC, and ablations validate the design choices, including layer choice and the fusion strategy. The work highlights the untapped value of pre-pooling statistics for robust OOD detection and outlines avenues for extending the framework to transformer-based architectures.

Abstract

Out-of-distribution (OOD) detection is critical for the safe deployment of deep neural networks. State-of-the-art post-hoc methods typically derive OOD scores from the output logits or penultimate feature vector obtained via global average pooling (GAP). We contend that this exclusive reliance on the logit or feature vector discards a rich, complementary signal: the raw channel-wise statistics of the pre-pooling feature map lost in GAP. In this paper, we introduce Catalyst, a post-hoc framework that exploits these under-explored signals. Catalyst computes an input-dependent scaling factor () on-the-fly from these raw statistics (e.g., mean, standard deviation, and maximum activation). This is then fused with the existing baseline score, multiplicatively modulating it -- an ``elastic scaling'' -- to push the ID and OOD distributions further apart. We demonstrate Catalyst is a generalizable framework: it seamlessly integrates with logit-based methods (e.g., Energy, ReAct, SCALE) and also provides a significant boost to distance-based detectors like KNN. As a result, Catalyst achieves substantial and consistent performance gains, reducing the average False Positive Rate by 32.87 on CIFAR-10 (ResNet-18), 27.94% on CIFAR-100 (ResNet-18), and 22.25% on ImageNet (ResNet-50). Our results highlight the untapped potential of pre-pooling statistics and demonstrate that Catalyst is complementary to existing OOD detection approaches.
Paper Structure (48 sections, 2 theorems, 30 equations, 10 figures, 27 tables)

This paper contains 48 sections, 2 theorems, 30 equations, 10 figures, 27 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Methodology
  4. Computing the Scaling Factor $\gamma$
  5. Elastic Scaling of the Score
  6. Generalizability of ${\texttt{Catalyst}}$
  7. Experiments
  8. CIFAR Evaluation
  9. ImageNet Evaluation
  10. Synergy with Existing Baselines
  11. Generalizability to Distance-Based Methods
  12. Hyperparameter Selection
  13. Comparison with Other Baselines
  14. Accuracy and Computational Overhead
  15. Ablation Study
  16. ...and 33 more sections

Key Result

Theorem 1

Under Assumptions assumption: scaling pattern and assumption: approx uncorrelation, the distributional separation of the multiplicatively scaled score, $S^{*}(\mathbf{x})$, is at least as great as that of the original score, $S(\mathbf{x})$, i.e., $\Delta_{\text{scaled}} \ge \Delta_{\text{original}}

Figures (10)

  • Figure 1: Information cues from each channel before the penultimate layer of a ResNet-50 trained on ImageNet-1k, evaluated with Texture as the OOD dataset. The x-axis shows channel indices; the y-axis shows cue strength. Left to right: (a) $\mu(\mathbf{x})$: mean activation, (b) $\sigma(\mathbf{x})$: standard deviation, (c) $\max(\mathbf{x})$: dominant activation, and (d) $H(\mathbf{x})$: entropy per channel.
  • Figure 2: Illustration of ${\texttt{Catalyst}}$'s effectiveness. The model is ResNet-50 trained on ImageNet-1k, evaluated on Texture (OOD). Here, we apply $\gamma$ computed from the channel-maximum statistic (m) multiplicatively to the baseline ReAct. (a) The unscaled score distribution shows more significant overlap than (b) the ${\texttt{Catalyst}}$-scaled score distribution.
  • Figure 3: Sensitivity analysis of the clipping percentile ($p$) on ${\texttt{Catalyst}}(m)$ performance. All values averaged over 4 OOD test datasets for a ResNet-50 (ImageNet).
  • Figure 4: Distribution of scaling factor $\gamma$ from the penultimate layer of a ResNet-50 trained on ImageNet-1k, evaluated with Texture as the OOD dataset. The scales show clear separation between ID and OOD samples. Left to right: (a) $\mu(\mathbf{x})$: mean, (b) $\sigma(\mathbf{x})$: standard deviation, (c) $\max(\mathbf{x})$: max
  • Figure 5: Distributions of the scaling factor $\gamma$, derived from the penultimate layer of a MobileNet-V2 model trained on ImageNet-1k. The rows (top to bottom) correspond to the OOD datasets: SUN, Places365, Texture, and iNaturalist. The columns (left to right) correspond to the statistical cue used to compute $\gamma$: (a) mean: $\mu(\mathbf{x})$, (b) standard deviation: $\sigma(\mathbf{x})$, and (c) maximum value: $\max(\mathbf{x})$ ( we used $\max(\mathbf{x})$ and $m(\mathbf{x})$ interchangeably). A clear pattern emerges: the distributions for the scene-based datasets (SUN, Places365) exhibit a significantly greater overlap with the in-distribution data compared to the more distinct Texture and iNaturalist datasets. This effect is particularly prominent when using the standard deviation $\sigma(\mathbf{x})$ and maximum value $\max(\mathbf{x})$ as information cues. We observe a similar pattern for ResNet-34 and ResNet-50 backbones. However, we also find that the overlap is more prominent for MobileNet-V2 than for ResNet-34, and in turn, more prominent for ResNet-34 than for ResNet-50.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Theorem 2
  • proof