Score Combining for Contrastive OOD Detection

TL;DR

The paper tackles unsupervised OOD detection by fusing multiple self-supervised scores through a principled generalized likelihood ratio test (GLRT). It formulates the negative-means problem on z-values derived from empirical, self-supervised scores, derives the GLRT statistic $t_{\mathsf{GLRT}}(\mathbf{z})$, and provides conformal-p-value-based false-alarm guarantees. By applying GLRT to CSI and SupCSI, it demonstrates consistent improvements over state-of-the-art baselines and classical p-value combination rules across dataset-vs-dataset and leave-one-class-out benchmarks. The approach also explores extensions with alternative covariance structures and supplementary scores, highlighting both performance gains and the sensitivity to score selection in practical OOD tasks.

Abstract

In out-of-distribution (OOD) detection, one is asked to classify whether a test sample comes from a known inlier distribution or not. We focus on the case where the inlier distribution is defined by a training dataset and there exists no additional knowledge about the novelties that one is likely to encounter. This problem is also referred to as novelty detection, one-class classification, and unsupervised anomaly detection. The current literature suggests that contrastive learning techniques are state-of-the-art for OOD detection. We aim to improve on those techniques by combining/ensembling their scores using the framework of null hypothesis testing and, in particular, a novel generalized likelihood ratio test (GLRT). We demonstrate that our proposed GLRT-based technique outperforms the state-of-the-art CSI and SupCSI techniques from Tack et al. 2020 in dataset-vs-dataset experiments with CIFAR-10, SVHN, LSUN, ImageNet, and CIFAR-100, as well as leave-one-class-out experiments with CIFAR-10. We also demonstrate that our GLRT outperforms the score-combining methods of Fisher, Bonferroni, Simes, Benjamini-Hochwald, and Stouffer in our application.

Figures (5)

• Figure 1: Left: Histogram of CSI's "norm" score $\{s_1(\boldsymbol{x}_i)\}_{i=1}^n$ (top) and "cos" score $\{s_2(\boldsymbol{x}_i)\}_{i=1}^n$ (bottom) for CIFAR-10 inliers $\mathcal{X}_\mathsf{test}^{\mathsf{cifar10}}$. Center: Histograms of the corresponding empirical z-values $\{\widehat{z}_1(s_1(\boldsymbol{x}_i))\}_{i=1}^n$ and $\{\widehat{z}_2(s_2(\boldsymbol{x}_i))\}_{i=1}^n$ with $\widehat{z}_1(\cdot)$ and $\widehat{z}_2(\cdot)$ constructed using CIFAR-10 inliers $\mathcal{X}_\mathsf{train}^{\mathsf{cifar10}}$. Right: Scatter plot of $\widehat{z}_1(s_1(\boldsymbol{x}_i))$ (horizontal axis) versus $\widehat{z}_2(s_2(\boldsymbol{x}_i))$ (vertical axis) for CIFAR-10 inliers $\mathcal{X}_\mathsf{test}^{\mathsf{cifar10}}$ and SVHN novelties $\mathcal{X}_\mathsf{test}^{\mathsf{svhn}}$. Contours show the value of the proposed GLRT score $t_{\mathsf{GLRT}}(\boldsymbol{z})$ versus $z_1$ and $z_2$.
• Figure 2: Left: Plot of $t_{\mathsf{GLRT}}(z)$ versus $z\in{\mathbb{R}}$ for $\epsilon\in\{0,0.1,0.2,0.3\}$, along with $t_{\mathsf{stouffer}}(z)=z$ and $t_{\mathsf{fisher}}(z)=\ln\Phi(z)$. Right: a plot of the same test statistics after shifting and scaling $t(z)$ so that $\Pr\{t(z)\leq 0\,|\,H_0\}=0.1$ and $t'(z_0)=1$ for $z_0$ such that $t(z_0)=0$.
• Figure 3: Examples of the beta pdf from (\ref{['eq:beta']}) and thresholds $\mathsf{a}$ that guarantee a target false-alarm rate (FAR) of at most $\alpha=0.05$ (red dotted line) with failure probability of at most $\delta=0.1$. The achieved false alarm rate, $\alpha_{\min}$, is shown by the dashed red line.
• Figure 4: AUROC of glrt-SupCSI+ vs. GLRT parameter $\epsilon$ for CIFAR-10 inliers and CIFAR-100 novelties.
• Figure 5: AUROC of eigen-score $t_{\mathsf{eig},k}(\boldsymbol{z})$ versus eigenvalue $\lambda_k$ for $k=1,\dots,m$ on two $m=24$ score experiments from Section \ref{['sec:experiments']}: CIFAR-10 inliers with SVHN novelties (left) and SVHN novelties with CIFAR-10 inliers (right).