Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection

TL;DR

Out-of-distribution detection remains challenging when OOD samples resemble in-distribution data. The paper introduces P-OCS, which performs a single perturbation confined to the orthogonal complement of the ID principal subspace in penultimate features, with the OOD score $s(x)=\sum_{t=0}^{T-1} \| z_{t+1}-z_t\|_2$ capturing accumulated drift. Theoretical justification shows one-step suffices in the small-perturbation regime, and empirical results on dermatology and ImageNet benchmarks demonstrate state-of-the-art performance with negligible computational overhead and no retraining. This orthogonal-complement perspective provides a clear geometric interpretation of OOD behavior, offering robust near-OOD and far-OOD detection across architectures and datasets. The approach is practical for real-time and safety-critical applications, and it opens avenues for extending to nonlinear subspaces and multi-modal settings.

Abstract

Out-of-distribution (OOD) detection is essential for deploying deep learning models in open-world environments. Existing approaches, such as energy-based scoring and gradient-projection methods, typically rely on high-dimensional representations to separate in-distribution (ID) and OOD samples. We introduce P-OCS (Perturbations in the Orthogonal Complement Subspace), a lightweight and theoretically grounded method that operates in the orthogonal complement of the principal subspace defined by ID features. P-OCS applies a single projected perturbation restricted to this complementary subspace, enhancing subtle ID-OOD distinctions while preserving the geometry of ID representations. We show that a one-step update is sufficient in the small-perturbation regime and provide convergence guarantees for the resulting detection score. Experiments across multiple architectures and datasets demonstrate that P-OCS achieves state-of-the-art OOD detection with negligible computational cost and without requiring model retraining, access to OOD data, or changes to model architecture.

Figures (11)

• Figure 1: PCA projection of ID and Near-OOD samples. ID samples (green) concentrate along the principal components, while Near-OOD samples (blue) exhibit greater dispersion across orthogonal directions. This illustrates the challenge in separating Near-OOD samples from ID using standard methods.
• Figure 2: Further PCA analysis highlighting the challenge of distinguishing Near-OOD from ID samples. The right plot shows how Near-OOD samples (blue) are dispersed across the feature space, making it difficult for traditional models to separate them from ID (green) samples without additional processing.
• Figure 4: ID vs OOD explained variance ratio (first 80 components) — on ID basis.
• Figure 5: ID vs OOD explained variance ratio in complement space (first 80 components).
• Figure 6: Dermatology results with a metric-wise layout (x-axis: AUROC, AUPR, FPR@95). P-OCS consistently leads across all metrics, combining stronger discrimination and precision–recall performance with substantially reduced high-recall false positives.