Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying Epistemic Uncertainty in Diffusion Models

Aditi Gupta, Raphael A. Meyer, Yotam Yaniv, Elynn Chen, N. Benjamin Erichson

TL;DR

This work tackles the challenge of quantifying epistemic uncertainty in diffusion models by separating it from diffusion-driven aleatoric noise. It introduces a Fisher–Laplace projection that maps parameter uncertainty into data space and a scalable FLARE estimator that randomizes parameters across layers to propagate epistemic variance along the reverse diffusion trajectory. The key contributions are a closed-form one-step projection, a trajectory-wide propagation recursion, and theoretical guarantees for the randomized estimator, demonstrated by improved uncertainty-aware filtering on synthetic time-series tasks compared to BayesDiff and last-layer Laplace methods. The approach yields faithful, sample-level uncertainty diagnostics and enables more reliable filtering of generated data without retraining or multiple inference passes, with potential impact on robustness and reliability of diffusion-based generation.

Abstract

To ensure high quality outputs, it is important to quantify the epistemic uncertainty of diffusion models.Existing methods are often unreliable because they mix epistemic and aleatoric uncertainty. We introduce a method based on Fisher information that explicitly isolates epistemic variance, producing more reliable plausibility scores for generated data. To make this approach scalable, we propose FLARE (Fisher-Laplace Randomized Estimator), which approximates the Fisher information using a uniformly random subset of model parameters. Empirically, FLARE improves uncertainty estimation in synthetic time-series generation tasks, achieving more accurate and reliable filtering than other methods. Theoretically, we bound the convergence rate of our randomized approximation and provide analytic and empirical evidence that last-layer Laplace approximations are insufficient for this task.

Quantifying Epistemic Uncertainty in Diffusion Models

TL;DR

This work tackles the challenge of quantifying epistemic uncertainty in diffusion models by separating it from diffusion-driven aleatoric noise. It introduces a Fisher–Laplace projection that maps parameter uncertainty into data space and a scalable FLARE estimator that randomizes parameters across layers to propagate epistemic variance along the reverse diffusion trajectory. The key contributions are a closed-form one-step projection, a trajectory-wide propagation recursion, and theoretical guarantees for the randomized estimator, demonstrated by improved uncertainty-aware filtering on synthetic time-series tasks compared to BayesDiff and last-layer Laplace methods. The approach yields faithful, sample-level uncertainty diagnostics and enables more reliable filtering of generated data without retraining or multiple inference passes, with potential impact on robustness and reliability of diffusion-based generation.

Abstract

To ensure high quality outputs, it is important to quantify the epistemic uncertainty of diffusion models.Existing methods are often unreliable because they mix epistemic and aleatoric uncertainty. We introduce a method based on Fisher information that explicitly isolates epistemic variance, producing more reliable plausibility scores for generated data. To make this approach scalable, we propose FLARE (Fisher-Laplace Randomized Estimator), which approximates the Fisher information using a uniformly random subset of model parameters. Empirically, FLARE improves uncertainty estimation in synthetic time-series generation tasks, achieving more accurate and reliable filtering than other methods. Theoretically, we bound the convergence rate of our randomized approximation and provide analytic and empirical evidence that last-layer Laplace approximations are insufficient for this task.
Paper Structure (47 sections, 3 theorems, 69 equations, 9 figures, 4 tables, 1 algorithm)

This paper contains 47 sections, 3 theorems, 69 equations, 9 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries on Diffusion Models
  3. Formalizing the UQ Problem
  4. Related Work
  5. Method
  6. Aleatoric randomness and conditioning.
  7. Fisher Information-guided Projection
  8. Propagation through the Denoising Steps
  9. Computational considerations.
  10. Randomized Subnetwork Approximation
  11. Theoretical guarantees.
  12. Approximating the Hessian.
  13. Diagnostics
  14. Relation to BayesDiff
  15. Theory
  16. ...and 32 more sections

Key Result

Proposition 1

Under a local Gaussian posterior $\theta \sim \mathcal{N}(\hat{\theta},{\bm{\Sigma}}_\theta)$, independence of the reverse-step noise $\eta_t$ from the model parameters $\theta$, and a first-order (delta-method) linearization of the one-step conditional mean around $\hat{\theta}$, the conditional co

Figures (9)

  • Figure 1: Mode interpolation in a 2D Gaussian mixture adapted from aithal2024understandingjazbec2025generative. The dataset consists of nine Gaussian modes arranged on a square grid. Top: uncertainty scores assigned to generated samples by BayesDiff (left), last-layer Laplace (LLLA; middle), and our method (right). Bottom: the same samples after filtering using a fixed uncertainty threshold. BayesDiff assigns low uncertainty to samples between modes, while LLLA further suppresses uncertainty due to its restriction to the final layer. In contrast, our method assigns high epistemic uncertainty in low-density regions between modes, enabling reliable removal of low-confidence samples.
  • Figure 2: Generated time-series samples before and after uncertainty-based filtering. Panel (a) shows the sinusoidal dataset and panel (b) the chirp dataset. Top: generated trajectories, colored by epistemic uncertainty (blue = low, red = high). Bottom: trajectories retained after filtering by uncertainty. Filtering removes implausible, off-manifold samples while preserving diverse, on-distribution trajectories.
  • Figure 4: DDIM - Mode interpolation on a 2D Gaussian mixture adapted from aithal2024understanding and jazbec2025generative. The dataset consists of 9 Gaussian modes arranged on a square grid. The top row shows uncertainty scores from BayesDiff (left), LLLA (middle), and our method (right) for generated samples. The bottom row shows the same generated samples after filtering by a fixed uncertainty threshold. BayesDiff fails to assign high scores to points between modes, while LLLA performs even worse, leading to unreliable filtering. In contrast, our method produces faithful uncertainty estimates, enabling consistent removal of low-confidence samples.
  • Figure 5: Parameter–budget ablation with FullSubnetLaplace. We retain $\{1\%, 5\%, 10\%, 30\%, 50\%\}$ of parameters. As the kept fraction increases, trajectories tighten around the data manifolds and the epistemic covariance $b_t^{2}\mathbf{J}_t\boldsymbol{\Sigma}_\theta\mathbf{J}_t^\top$ contracts smoothly; $10$–$30\%$ already yields stable mode coverage, with diminishing returns beyond $30$–$50\%$.
  • Figure 6: Four-way epistemic UQ comparison in the full-curvature regime. We compare uncertainty estimates from (left to right): BayesDiff (last-layer Laplace, total recursion), Fuller-Hessian Laplace (Bayesdiff recursion), BayesDiff (epistemic-only recursion), and FLARE (Fisher--Laplace projection). In this controlled setting, full-curvature computation is feasible and allows direct inspection of epistemic structure. The scalable approximation of kou2023bayesdiff (BayesDiff) already breaks down in this regime, producing uncertainty maps that are not epistemically meaningful relative to the full-Hessian reference, whereas FLARE closely tracks the curvature-based baseline while avoiding full-Hessian computation.
  • ...and 4 more figures

Theorems & Definitions (7)

  • Proposition 1: One-step Fisher--Laplace projection
  • Lemma 1
  • Theorem 1
  • proof
  • proof
  • Remark 1
  • proof