EigenScore: OOD Detection using Covariance in Diffusion Models

TL;DR

Problem: OOD detection in diffusion-model pipelines is critical for safety, yet existing likelihood or score-based metrics can be unstable or misorder inputs. Approach: EigenScore uses the eigen-spectrum of the denoising posterior covariance, $Cov_p[{\bm{x}}|{\bm{x}}_t]$, to quantify distribution shift, estimated via a Jacobian-free subspace iteration on the forward denoiser. Contributions: a theoretical link between distribution shift and excess denoising error through the posterior covariance, a practical method to compute leading eigenvalues without explicit Jacobians, and strong empirical AUROC gains including in near-OOD settings. Significance: delivers a principled, scalable OOD detector for diffusion-based systems with real-world impact in safety-critical tasks.

Abstract

Out-of-distribution (OOD) detection is critical for the safe deployment of machine learning systems in safety-sensitive domains. Diffusion models have recently emerged as powerful generative models, capable of capturing complex data distributions through iterative denoising. Building on this progress, recent work has explored their potential for OOD detection. We propose EigenScore, a new OOD detection method that leverages the eigenvalue spectrum of the posterior covariance induced by a diffusion model. We argue that posterior covariance provides a consistent signal of distribution shift, leading to larger trace and leading eigenvalues on OOD inputs, yielding a clear spectral signature. We further provide analysis explicitly linking posterior covariance to distribution mismatch, establishing it as a reliable signal for OOD detection. To ensure tractability, we adopt a Jacobian-free subspace iteration method to estimate the leading eigenvalues using only forward evaluations of the denoiser. Empirically, EigenScore achieves SOTA performance, with up to 5% AUROC improvement over the best baseline. Notably, it remains robust in near-OOD settings such as CIFAR-10 vs CIFAR-100, where existing diffusion-based methods often fail.

Figures (3)

• Figure 1: We compare negative log-likelihood (NLL), score norm $\sqrt{\sum_t \|\epsilon_\theta({\bm{x}}_t, t)\|_2^2}$, score derivative norm $\sqrt{\sum_t \|\partial_t \epsilon_\theta({\bm{x}}_t, t)\|_2^2}$, and the eigenvalue sum (ours) $\sum_{t,k}\lambda_k^t({\bm{x}}_t)$ as OOD detection statistics. Top row: near OOD task for C10 (InD) vs. C100, NLL and score-based metrics fail to separate distributions, showing substantial overlap. Bottom row: for C10 (InD) vs. SVHN (OOD), the ordering of metrics inverts—score and derivative norms assign lower values to OOD than InD, making thresholds unreliable. In both settings, our eigenvalue-based metric achieves clear separation and consistently assigns higher scores to OOD samples.
• Figure 2: Denoised outputs (left), corresponding uncertainty maps (first principle component) (middle), and violin plots of the three largest eigenvalues for CelebA dataset (right). Top: clean CelebA image and its noisy variants for varying t. Middle: InD model (trained on CelebA) applied to CelebA inputs. Bottom: OOD model (trained on C100) applied to the same inputs. InD models yield sharp reconstructions and localized uncertainty with smaller leading eigenvalues, whereas OOD models produce blurrier outputs, diffuse uncertainty, and inflated eigenvalues—highlighting the eigenvalue spectrum as an indicator of distribution shift.
• Figure 3: Ablation on eigenvalue informativeness across $t$. Performance declines with increasing noise, consistent with Lem. \ref{['lem:noise']}, while $\lambda_1$ retains the strongest OOD signal compared to $\lambda_2$ and $\lambda_3$, supporting Prop. \ref{['prop:kyfan']}.

Theorems & Definitions (5)

• Proposition 1
• Lemma 1
• Proposition 2: Ky Fan’s theorem fan1950theorem
• proof
• proof