Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Breaking Semantic Hegemony: Decoupling Principal and Residual Subspaces for Generalized OOD Detection

Ningkang Peng, Xiaoqian Peng, Yuhao Zhang, Qianfeng Yu, Feng Xing, Peirong Ma, Xichen Yang, Yi Chen, Tingyu Lu, Yanhui Gu

TL;DR

D-KNN, a training-free, plug-and-play geometric decoupling framework, utilizes orthogonal decomposition to explicitly separate semantic components from structural residuals and introduces a dual-space calibration mechanism to reactivate the model's sensitivity to weak residual signals.

Abstract

While feature-based post-hoc methods have made significant strides in Out-of-Distribution (OOD) detection, we uncover a counter-intuitive Simplicity Paradox in existing state-of-the-art (SOTA) models: these models exhibit keen sensitivity in distinguishing semantically subtle OOD samples but suffer from severe Geometric Blindness when confronting structurally distinct yet semantically simple samples or high-frequency sensor noise. We attribute this phenomenon to Semantic Hegemony within the deep feature space and reveal its mathematical essence through the lens of Neural Collapse. Theoretical analysis demonstrates that the spectral concentration bias, induced by the high variance of the principal subspace, numerically masks the structural distribution shift signals that should be significant in the residual subspace. To address this issue, we propose D-KNN, a training-free, plug-and-play geometric decoupling framework. This method utilizes orthogonal decomposition to explicitly separate semantic components from structural residuals and introduces a dual-space calibration mechanism to reactivate the model's sensitivity to weak residual signals. Extensive experiments demonstrate that D-KNN effectively breaks Semantic Hegemony, establishing new SOTA performance on both CIFAR and ImageNet benchmarks. Notably, in resolving the Simplicity Paradox, it reduces the FPR95 from 31.3% to 2.3%; when addressing sensor failures such as Gaussian noise, it boosts the detection performance (AUROC) from a baseline of 79.7% to 94.9%.

Breaking Semantic Hegemony: Decoupling Principal and Residual Subspaces for Generalized OOD Detection

TL;DR

D-KNN, a training-free, plug-and-play geometric decoupling framework, utilizes orthogonal decomposition to explicitly separate semantic components from structural residuals and introduces a dual-space calibration mechanism to reactivate the model's sensitivity to weak residual signals.

Abstract

While feature-based post-hoc methods have made significant strides in Out-of-Distribution (OOD) detection, we uncover a counter-intuitive Simplicity Paradox in existing state-of-the-art (SOTA) models: these models exhibit keen sensitivity in distinguishing semantically subtle OOD samples but suffer from severe Geometric Blindness when confronting structurally distinct yet semantically simple samples or high-frequency sensor noise. We attribute this phenomenon to Semantic Hegemony within the deep feature space and reveal its mathematical essence through the lens of Neural Collapse. Theoretical analysis demonstrates that the spectral concentration bias, induced by the high variance of the principal subspace, numerically masks the structural distribution shift signals that should be significant in the residual subspace. To address this issue, we propose D-KNN, a training-free, plug-and-play geometric decoupling framework. This method utilizes orthogonal decomposition to explicitly separate semantic components from structural residuals and introduces a dual-space calibration mechanism to reactivate the model's sensitivity to weak residual signals. Extensive experiments demonstrate that D-KNN effectively breaks Semantic Hegemony, establishing new SOTA performance on both CIFAR and ImageNet benchmarks. Notably, in resolving the Simplicity Paradox, it reduces the FPR95 from 31.3% to 2.3%; when addressing sensor failures such as Gaussian noise, it boosts the detection performance (AUROC) from a baseline of 79.7% to 94.9%.
Paper Structure (16 sections, 1 theorem, 11 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 16 sections, 1 theorem, 11 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generalized Post-hoc OOD Detection
  4. Distance-based and Subspace Methods
  5. Neural Collapse and Feature Geometry
  6. Methodology
  7. Motivation and Preliminaries
  8. Dual-Space Geometric Decoupling
  9. Dual-Space Calibration
  10. Algorithm Summary
  11. Theoretical Analysis: Asymptotic Separability
  12. Experiments
  13. Experimental Setup
  14. Experimental Results and Analysis
  15. Analysis and Discussion
  16. ...and 1 more sections

Key Result

Theorem 1

Under the ideal NC regime ($\sigma_{\text{in}} \to 0$), assuming a fundamental gap exists such that $\mu_{\text{out}} - \mu_{\text{in}} \ge \Delta > 0$, the detection risk $\mathcal{R}$ for the calibrated score $\tilde{S}$ vanishes asymptotically.

Figures (4)

  • Figure 1: Performance comparison of OOD detection methods on CIFAR-100. We categorize existing methods into three paradigms: Logit-based (pink squares), Distance-based (purple triangles), and Subspace-based (green circles).
  • Figure 2: Eigenspectrum Analysis and the Semantic Hegemony Ratio $\rho$. This figure illustrates the exponential decay of eigenvalues across dimensions, revealing the severity of feature energy collapse into the principal subspace under various architectures.
  • Figure 3: Visualization Comparison of OOD Sample Distributions in Original and Residual Spaces. This scatter plot clearly demonstrates how OOD samples incur semantic overlap within the original space and how geometric decoupling and separation are realized through the residual space.
  • Figure 4: Sensitivity Analysis of Detection Performance to the Fusion Weight Parameter $\alpha$ across Different Datasets. This line chart demonstrates that D-KNN exhibits a stable inverted U-shaped performance distribution across various tasks, reflecting the method's low dependence on rigorous parameter tuning.

Theorems & Definitions (2)

  • Theorem 1: Asymptotic Risk Vanishing
  • proof