Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Out-of-Distribution Detection with a Single Unconditional Diffusion Model

Alvin Heng, Alexandre H. Thiery, Harold Soh

TL;DR

A novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal is introduced, which uses a single diffusion model originally trained to perform unconditional generation for OOD detection.

Abstract

Out-of-distribution (OOD) detection is a critical task in machine learning that seeks to identify abnormal samples. Traditionally, unsupervised methods utilize a deep generative model for OOD detection. However, such approaches require a new model to be trained for each inlier dataset. This paper explores whether a single model can perform OOD detection across diverse tasks. To that end, we introduce Diffusion Paths (DiffPath), which uses a single diffusion model originally trained to perform unconditional generation for OOD detection. We introduce a novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal. Extensive experiments show that with a single model, DiffPath is competitive with prior work using individual models on a variety of OOD tasks involving different distributions. Our code is publicly available at https://github.com/clear-nus/diffpath.

Out-of-Distribution Detection with a Single Unconditional Diffusion Model

TL;DR

A novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal is introduced, which uses a single diffusion model originally trained to perform unconditional generation for OOD detection.

Abstract

Out-of-distribution (OOD) detection is a critical task in machine learning that seeks to identify abnormal samples. Traditionally, unsupervised methods utilize a deep generative model for OOD detection. However, such approaches require a new model to be trained for each inlier dataset. This paper explores whether a single model can perform OOD detection across diverse tasks. To that end, we introduce Diffusion Paths (DiffPath), which uses a single diffusion model originally trained to perform unconditional generation for OOD detection. We introduce a novel technique of measuring the rate-of-change and curvature of the diffusion paths connecting samples to the standard normal. Extensive experiments show that with a single model, DiffPath is competitive with prior work using individual models on a variety of OOD tasks involving different distributions. Our code is publicly available at https://github.com/clear-nus/diffpath.
Paper Structure (33 sections, 2 theorems, 23 equations, 3 figures, 8 tables, 1 algorithm)

This paper contains 33 sections, 2 theorems, 23 equations, 3 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Score-based Diffusion Models.
  4. Unsupervised OOD Detection.
  5. Diffusion Models for OOD Detection
  6. Diffusion Model Likelihoods Do Not Work for OOD Detection
  7. Scores as an OOD Statistic
  8. Scores as KL Divergence Proxy.
  9. Score as First-Order Taylor Expansion.
  10. Second-Order Taylor Expansion (DiffPath-1D)
  11. Connections to Optimal Transport
  12. Higher-dimensional Statistic (DiffPath-6D)
  13. Related Works
  14. Experiments
  15. Datasets.
  16. ...and 18 more sections

Key Result

Theorem 1

Denote $\phi_t$ and $\psi_t$ as the marginals from evolving two distinct distributions $\phi_0$ and $\psi_0$ via their respective probability flow ODEs (Eq. eq:pf_ode_reparam) forward in time. We consider the case with the same forward process, i.e., the two PF ODEs have the same ${\bm{f}}({\mathbf{

Figures (3)

  • Figure 1: Illustration of the diffusion paths of samples from two different distributions (CIFAR10 and SVHN) obtained via DDIM integration. The paths have different first and second derivatives (rate-of-change and curvature). We propose to measure these quantities for OOD detection.
  • Figure 2: Histograms of various statistics of the respective training sets. The NLL is calculated using a diffusion model trained on CIFAR10, while the other two statistics are calculated with a model trained on ImageNet.
  • Figure 3: Illustration of the forward integration of Eq. \ref{['eq:ddim_ode']} on samples from CIFAR10, SVHN and CelebA. Both the ImageNet and CelebA models are able bring the samples approximately to standard normal. Other than the case where the CelebA model is used to integrate CelebA samples (last row of the right figure), the samples shown here have not been seen by the models during training. While in certain cases the end result appears to contain features of the original image, thus deviating from an isotropic Gaussian (e.g., first row of the right figure), empirically we find that the scores remain accurate enough for outlier detection; see Sec. \ref{['sec:exp']} for quantitative results.

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 1
  • proof