On Stability and Robustness of Diffusion Posterior Sampling for Bayesian Inverse Problems
Yiming Yang, Xiaoyuan Cheng, Yi He, Kaiyu Li, Wenxuan Yuan, Zhuo Sun
TL;DR
This work analyzes diffusion-prior methods for Bayesian inverse problems, proving that diffusion posterior sampling (DPS) is stable with respect to measurement noise but can be fragile under likelihood misspecification. To address this, the authors introduce Robust Diffusion Posterior (RDP), a modular generalized Bayesian framework that adaptively down-weights problematic measurements via an IMQ-based per-coordinate weighting, yielding a uniformly bounded posterior influence function. Theoretical results show DPS stability and bound the posterior error under noise, while RDP provides robust guarantees with a quantified bias that vanishes as the robustness parameter grows. Empirically, RDP improves robustness across diverse scientific IPs and natural-image tasks, maintaining performance under well-specified likelihoods and substantially mitigating artifacts under heavy-tailed noise and outliers. The approach is readily compatible with existing gradient-based diffusion samplers, offering a practical path to robust Bayesian inference in high-dimensional inverse problems.
Abstract
Diffusion models have recently emerged as powerful learned priors for Bayesian inverse problems (BIPs). Diffusion-based solvers rely on a presumed likelihood for the observations in BIPs to guide the generation process. However, the link between likelihood and recovery quality for BIPs is unclear in previous works. We bridge this gap by characterizing the posterior approximation error and proving the \emph{stability} of the diffusion-based solvers. Meanwhile, an immediate result of our findings on stability demonstrates the lack of robustness in diffusion-based solvers, which remains unexplored. This can degrade performance when the presumed likelihood mismatches the unknown true data generation processes. To address this issue, we propose a simple yet effective solution, \emph{robust diffusion posterior sampling}, which is provably \emph{robust} and compatible with existing gradient-based posterior samplers. Empirical results on scientific inverse problems and natural image tasks validate the effectiveness and robustness of our method, showing consistent performance improvements under challenging likelihood misspecifications.