Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How many measurements are enough? Bayesian recovery in inverse problems with general distributions

Ben Adcock, Nick Huang

TL;DR

This work provides non-asymptotic Bayesian guarantees for recovering signals in inverse problems with highly general priors, forward operators, and noise processes by linking recovery success to the approximate covering number Cov_{η,δ}(P) and concentration bounds. It demonstrates that for Lipschitz generative priors, the sample complexity scales log-linearly with the latent dimension k, yielding near-optimal m = O(k log k) measurements, while for more structured forward models like randomly subsampled orthogonal transforms, coherence μ(U;D) governs m. The results cover both subgaussian and practical forward models (e.g., Fourier-based MRI), unify Bayesian recovery with classical compressed sensing ideas, and address stability against non-Gaussian noise and potential DL hallucinations through rigorous guarantees. The framework highlights the central roles of model complexity via Cov and of coherence/concentration in achieving reliable Bayesian inverse problem recovery, and points to future work on improving bounds and extending to broader noise models and efficient posterior sampling.

Abstract

We study the sample complexity of Bayesian recovery for solving inverse problems with general prior, forward operator and noise distributions. We consider posterior sampling according to an approximate prior $\mathcal{P}$, and establish sufficient conditions for stable and accurate recovery with high probability. Our main result is a non-asymptotic bound that shows that the sample complexity depends on (i) the intrinsic complexity of $\mathcal{P}$, quantified by its so-called approximate covering number, and (ii) concentration bounds for the forward operator and noise distributions. As a key application, we specialize to generative priors, where $\mathcal{P}$ is the pushforward of a latent distribution via a Deep Neural Network (DNN). We show that the sample complexity scales log-linearly with the latent dimension $k$, thus establishing the efficacy of DNN-based priors. Generalizing existing results on deterministic (i.e., non-Bayesian) recovery for the important problem of random sampling with an orthogonal matrix $U$, we show how the sample complexity is determined by the coherence of $U$ with respect to the support of $\mathcal{P}$. Hence, we establish that coherence plays a fundamental role in Bayesian recovery as well. Overall, our framework unifies and extends prior work, providing rigorous guarantees for the sample complexity of solving Bayesian inverse problems with arbitrary distributions.

How many measurements are enough? Bayesian recovery in inverse problems with general distributions

TL;DR

This work provides non-asymptotic Bayesian guarantees for recovering signals in inverse problems with highly general priors, forward operators, and noise processes by linking recovery success to the approximate covering number Cov_{η,δ}(P) and concentration bounds. It demonstrates that for Lipschitz generative priors, the sample complexity scales log-linearly with the latent dimension k, yielding near-optimal m = O(k log k) measurements, while for more structured forward models like randomly subsampled orthogonal transforms, coherence μ(U;D) governs m. The results cover both subgaussian and practical forward models (e.g., Fourier-based MRI), unify Bayesian recovery with classical compressed sensing ideas, and address stability against non-Gaussian noise and potential DL hallucinations through rigorous guarantees. The framework highlights the central roles of model complexity via Cov and of coherence/concentration in achieving reliable Bayesian inverse problem recovery, and points to future work on improving bounds and extending to broader noise models and efficient posterior sampling.

Abstract

We study the sample complexity of Bayesian recovery for solving inverse problems with general prior, forward operator and noise distributions. We consider posterior sampling according to an approximate prior , and establish sufficient conditions for stable and accurate recovery with high probability. Our main result is a non-asymptotic bound that shows that the sample complexity depends on (i) the intrinsic complexity of , quantified by its so-called approximate covering number, and (ii) concentration bounds for the forward operator and noise distributions. As a key application, we specialize to generative priors, where is the pushforward of a latent distribution via a Deep Neural Network (DNN). We show that the sample complexity scales log-linearly with the latent dimension , thus establishing the efficacy of DNN-based priors. Generalizing existing results on deterministic (i.e., non-Bayesian) recovery for the important problem of random sampling with an orthogonal matrix , we show how the sample complexity is determined by the coherence of with respect to the support of . Hence, we establish that coherence plays a fundamental role in Bayesian recovery as well. Overall, our framework unifies and extends prior work, providing rigorous guarantees for the sample complexity of solving Bayesian inverse problems with arbitrary distributions.
Paper Structure (31 sections, 22 theorems, 198 equations)

This paper contains 31 sections, 22 theorems, 198 equations.

Table of Contents

  1. Introduction
  2. Overview
  3. Examples
  4. Significance
  5. Related work
  6. Preliminaries
  7. Notation
  8. Approximate covering numbers
  9. Bounds for $\mathcal{A}$ and $\mathcal{E}$
  10. Main results
  11. Example: Subgaussian random matrices with Gaussian noise
  12. Example: Randomly-subsampled orthogonal transforms with Gaussian noise
  13. Covering number and sample complexity estimates
  14. Generative DNNs
  15. Distributions of sparse vectors
  16. ...and 16 more sections

Key Result

Theorem 1.1

Let $1 \leq p \leq \infty$, $0 < \delta \leq 1/4$, $\varepsilon,\eta > 0$ and suppose that conditions (i)--(v) hold with $W_p(\mathcal{R} , \mathcal{P}) \leq \varepsilon / (2 m \theta)$ and $\sigma \geq \varepsilon / \delta^{1/p}$. Suppose that $x^* \sim \mathcal{R}$, $A \sim \mathcal{A}$, $e \sim

Theorems & Definitions (56)

  • Theorem 1.1: Simplified main result
  • Theorem 1.2: Subgaussian measurement matrices, simplified
  • Theorem 1.3: Subsampled orthogonal transforms, simplified
  • Definition 2.1: Approximate covering number
  • remark 1: Relation to non-Bayesian compressed sensing
  • Definition 2.2: Concentration bounds for $\mathcal{A}$
  • Definition 2.3: Concentration bound for $\mathcal{E}$
  • Definition 2.4: Density shift bounds for $\mathcal{E}$
  • Theorem 3.1
  • remark 2: The concentration bounds in Theorem \ref{['t:main-res']}
  • ...and 46 more