Beyond Binary Out-of-Distribution Detection: Characterizing Distributional Shifts with Multi-Statistic Diffusion Trajectories

TL;DR

This work addresses the limitation of scalar OOD detection scores by introducing DISC, a diffusion-based method that builds a time-resolved, multi-statistic embedding from denoising trajectories to capture distributional shifts across multiple noise levels. By combining time-expanded reconstruction discrepancies with texture and frequency descriptors, DISC provides both competitive OOD detection and the ability to classify the type of OOD (e.g., covariate vs semantic shifts) on image and tabular data. The authors prove a theoretical non-identifiability result showing that any single scalar statistic cannot distinguish between certain alternative distributions, motivating the need for multi-dimensional statistics. Empirically, DISC matches or surpasses state-of-the-art detectors on ImageNet/CIFAR-10 and ADBench, achieving strong AUROC and improved OOD-type discrimination, signaling a move toward richer open-world evaluations and continual adaptation capabilities.

Abstract

Detecting out-of-distribution (OOD) data is critical for machine learning, be it for safety reasons or to enable open-ended learning. However, beyond mere detection, choosing an appropriate course of action typically hinges on the type of OOD data encountered. Unfortunately, the latter is generally not distinguished in practice, as modern OOD detection methods collapse distributional shifts into single scalar outlier scores. This work argues that scalar-based methods are thus insufficient for OOD data to be properly contextualized and prospectively exploited, a limitation we overcome with the introduction of DISC: Diffusion-based Statistical Characterization. DISC leverages the iterative denoising process of diffusion models to extract a rich, multi-dimensional feature vector that captures statistical discrepancies across multiple noise levels. Extensive experiments on image and tabular benchmarks show that DISC matches or surpasses state-of-the-art detectors for OOD detection and, crucially, also classifies OOD type, a capability largely absent from prior work. As such, our work enables a shift from simple binary OOD detection to a more granular detection.

Figures (6)

• Figure 1: Simplified illustration to highlight that standard out-of-distribution (OOD) detection methods typically fail to differentiate between the distinct types of shift. In the example, density or distance-based methods would collapse their scores and conflate the two kinds of OOD samples. Reasoning of failure modes for other methods follows analogously.
• Figure 2: OOD scores overlap significantly for different OOD types for models trained on CIFAR-10 as in-distribution. This observation is independent of the employed OOD detector (panels a-c).
• Figure 3: AUROC in the standard OOD detection setting across various OOD types remains high across detectors (x-axis) for models trained on CIFAR-10.
• Figure 4: OOD-type classification accuracy (using unsupervised K-means and supervised MLP approaches) is substantially worse when compared to traditional binary OOD-detection, as presented in Figure \ref{['fig:cifar:auroc']}.
• Figure 5: (a) Average SSIM vs. diffusion timestep. When trained on butterflies, SSIM highlights the structural separation of naturalistic shifts (e.g., flip, black occlusion) from in-distribution samples as denoising progresses. (b) Average LPIPS vs. diffusion timestep. Perceptual similarity score for different distribution shifts show different trends over time, revealing the importance of time-expanded information in capturing differences across OOD types.