Beyond Perceptual Distances: Rethinking Disparity Assessment for Out-of-Distribution Detection with Diffusion Models

TL;DR

This work points out two main limitations in DM-based OoD detection methods and looks into the deep representations from the classifier-under-protection with the novel metrics devised correspondingly, leading to more informative disparity assessments between InD and OoD.

Abstract

Out-of-Distribution (OoD) detection aims to justify whether a given sample is from the training distribution of the classifier-under-protection, i.e., In-Distribution (InD), or from OoD. Diffusion Models (DMs) are recently utilized in OoD detection by using the perceptual distances between the given image and its DM generation. DM-based methods bring fresh insights to the field, yet remain under-explored. In this work, we point out two main limitations in DM-based OoD detection methods: (i) the perceptual metrics on the disparities between the given sample and its generation are devised only at human-perceived levels, ignoring the abstract or high-level patterns that help better reflect the intrinsic disparities in distribution; (ii) only the raw image contents are taken to measure the disparities, while other representations, i.e., the features and probabilities from the classifier-under-protection, are easy to access at hand but are ignored. To this end, our proposed detection framework goes beyond the perceptual distances and looks into the deep representations from the classifier-under-protection with our novel metrics devised correspondingly, leading to more informative disparity assessments between InD and OoD. An anomaly-removal strategy is integrated to remove the abnormal OoD information in the generation, further enhancing the distinctiveness of disparities. Our work has demonstrated state-of-the-art detection performances among DM-based methods in extensive experiments.

Figures (4)

• Figure 1: Motivation for informative disparity assessments. (i) Given an InD image $\bm x_{\rm in}$, its DM generation $\bm{\hat{x}}_{\rm in}$ has a small disparity in distribution with $\bm x_{\rm in}$. However, due to the rich and unique image contents, a large perceptual distance is obtained between $\bm x_{\rm in}$ and $\bm {\hat{x}}_{\rm in}$, e.g., $\text{LPIPS}(\bm x_{\rm in}, \bm {\hat{x}}_{\rm in})$. (ii) Given a OoD image $\bm x_{\rm out}$, its DM generation $\bm{\hat{x}}_{\rm out}$ should have a large distribution disparity with $\bm x_{\rm out}$. However, due to the resembling local image structure and texture, a small perceptual distance, e.g., $\text{LPIPS}(\bm x_{\rm out}, \bm {\hat{x}}_{\rm out})$, is obtained, leading to unsatisfactory results. $\mathrm{D^3}$ leverages deep representations with our devised metrics, enhancing the separability in assessing InD and OoD.
• Figure 2: Schematic illustration of $\mathrm{D^3}$. The classifier-under-protection is introduced to calculate the distribution disparity between a test image $\bm {x}$ and its DM generation $\bm {\hat{x}}$ in the learned feature space and probability space. An anomaly-removal strategy removes the abnormal OoD information hidden in the DM-generated $\bm {\hat{x}}$ for more significant distribution disparities and enhanced detection performance.
• Figure 3: The average detection FPR and AUROC on multiple OoD datasets w.r.t varied coefficients $\lambda$ of $\mathrm{D^3}$.
• Figure 4: The average detection FPR and AUROC on multiple OoD datasets w.r.t varied diffusion time steps $T$ of $\mathrm{D^3}$.