Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Score-Based VAMP with Fisher-Information-Based Onsager Correction

Tadashi Wadayama, Takumi Takahashi

TL;DR

This work presents SC-VAMP, a score-based version of VAMP in which the Onsager correction is computed from conditional Fisher information, enabling a Jacobian-free, data-driven approach. By representing the posterior MMSE estimator via Tweedie’s formula and estimating the Onsager term from learned scores, SC-VAMP accommodates arbitrary priors and nonlinear forward models, especially when combined with random orthogonal/unitary mixing. The framework remains consistent with Bayes-optimal VAMP in the linear Gaussian limit and provides an information-theoretic perspective on decoupling via the entropic CLT. Data-driven denoisers learned through DSM, along with efficient Fisher information estimation and optional pre-trained denoisers, extend VAMP to complex, real-world inverse problems while retaining scalar SE tractability. Numerical experiments validate both analytic SE alignment in linear settings and efficacy with learned scores under correlated priors, underscoring practical applicability across nonlinear and structured sensing scenarios.

Abstract

We propose score-based VAMP (SC-VAMP), a variant of vector approximate message passing (VAMP) in which the Onsager correction is expressed and computed via conditional Fisher information, thereby enabling a Jacobian-free implementation. Using learned score functions, SC-VAMP constructs nonlinear MMSE estimators through Tweedie's formula and derives the corresponding Onsager terms from the score-norm statistics, avoiding the need for analytical derivatives of the prior or likelihood. When combined with random orthogonal/unitary mixing to mitigate non-ideal, structured or correlated sensing settings, the proposed framework extends VAMP to complex black-box inference problems where explicit modeling is intractable. Finally, by leveraging the entropic CLT, we provide an information-theoretic perspective on the Gaussian approximation underlying SE, offering insight into the decoupling principle beyond idealized i.i.d. settings, including nonlinear regimes.

Score-Based VAMP with Fisher-Information-Based Onsager Correction

TL;DR

This work presents SC-VAMP, a score-based version of VAMP in which the Onsager correction is computed from conditional Fisher information, enabling a Jacobian-free, data-driven approach. By representing the posterior MMSE estimator via Tweedie’s formula and estimating the Onsager term from learned scores, SC-VAMP accommodates arbitrary priors and nonlinear forward models, especially when combined with random orthogonal/unitary mixing. The framework remains consistent with Bayes-optimal VAMP in the linear Gaussian limit and provides an information-theoretic perspective on decoupling via the entropic CLT. Data-driven denoisers learned through DSM, along with efficient Fisher information estimation and optional pre-trained denoisers, extend VAMP to complex, real-world inverse problems while retaining scalar SE tractability. Numerical experiments validate both analytic SE alignment in linear settings and efficacy with learned scores under correlated priors, underscoring practical applicability across nonlinear and structured sensing scenarios.

Abstract

We propose score-based VAMP (SC-VAMP), a variant of vector approximate message passing (VAMP) in which the Onsager correction is expressed and computed via conditional Fisher information, thereby enabling a Jacobian-free implementation. Using learned score functions, SC-VAMP constructs nonlinear MMSE estimators through Tweedie's formula and derives the corresponding Onsager terms from the score-norm statistics, avoiding the need for analytical derivatives of the prior or likelihood. When combined with random orthogonal/unitary mixing to mitigate non-ideal, structured or correlated sensing settings, the proposed framework extends VAMP to complex black-box inference problems where explicit modeling is intractable. Finally, by leveraging the entropic CLT, we provide an information-theoretic perspective on the Gaussian approximation underlying SE, offering insight into the decoupling principle beyond idealized i.i.d. settings, including nonlinear regimes.
Paper Structure (56 sections, 1 theorem, 71 equations, 3 figures)

This paper contains 56 sections, 1 theorem, 71 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation
  4. Problem Setup
  5. Related Works
  6. Our Contributions
  7. Score-Based VAMP
  8. Score-Based Representation of SISO Modules
  9. Score-based VAMP (SC-VAMP) Algorithm
  10. Derivation of SISO Module
  11. Problem Setting
  12. Conditional Tweedie's Formula
  13. Derivation of Extrinsic Output with Onsager Correction
  14. Derivation of Extrinsic Variance $v_{out}$
  15. Derivation of Extrinsic Mean x_out
  16. ...and 41 more sections

Key Result

Theorem 1

Consider a scalar Gaussian channel $Y = X + Z$, where $X \sim \mathcal{N}(0, P)$ and noise $Z \sim \mathcal{N}(0, \sigma^2)$. The MI associated with the fixed point of SC-VAMP satisfies where $I_{\mathsf{VAMP}}$ denotes the MI corresponding to the SE fixed point.

Figures (3)

  • Figure 1: MSE convergence of SC-VAMP (actual) versus SE (theory) for a linear observation system with Bernoulli-Gaussian prior ($N=2000$, $M=1000$, $\rho=0.1$, SNR$=20$ dB, batch size $B=200$).
  • Figure 2: EXIT-style analysis of SC-VAMP showing Module A and Module B transfer characteristics, along with the SE trajectory (green) and actual SC-VAMP trajectory (gray dashed).
  • Figure 3: MSE convergence of SC-VAMP with learned pairwise score function for correlated 2D Gaussian prior ($N=2000$, $M=1000$, $\xi=0.9$, SNR$=20$ dB). The learned score network captures the pairwise correlation structure.

Theorems & Definitions (2)

  • Theorem 1: Optimality in Scalar Gaussian Channels
  • Remark 1: Generalization to Vector Linear Gaussian Models