Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vector Approximate Survey Propagation

Qun Chen, Haochuan Zhang, Huimin Zhu

TL;DR

Simulations reveal that VASP significantly surpasses VAMP and GASP in estimation accuracy, particularly when the assumed prior is discrete-supported and the measurement matrix is non-i.i.d., and can efficiently approximate the postulated MAP estimator with only cubic complexity.

Abstract

Approximate Message Passing (AMP), originally designed to solve high-dimensional linear inverse problems, has found broad applications in signal processing and statistical inference. Among its key variants, Vector Approximate Message Passing (VAMP) and Generalized Approximate Survey Propagation (GASP) have demonstrated effectiveness even in scenarios where the assumed generative models differ from the true models. However, the maximum a posteriori (MAP) versions of VAMP and GASP have limitations: VAMP is restricted to differentiable priors and likelihoods, while GASP requires the measurement matrix to have independent identically distributed (i.i.d.) elements. To overcome these limitations, this paper introduces a new algorithm, Vector Approximate Survey Propagation (VASP). VASP utilizes survey propagation to handle non-differentiable priors and likelihoods, along with employs vector-form messages to account for correlations in the measurement matrix. Simulations reveal that VASP significantly surpasses VAMP and GASP in estimation accuracy, particularly when the assumed prior is discrete-supported and the measurement matrix is non-i.i.d.. Additionally, the state evolution (SE) of VASP, derived heuristically, accurately reflects the per-iteration mean squared error (MSE). A comparison between the SE and the free energy computed by Takahashi and Kabashima under the one-step replica symmetry breaking (1RSB) ansatz shows that the SE's fixed-point equations align with the free energy's saddle point equations. This suggests that VASP efficiently implements the postulated MAP estimator (which is NP-hard in the worst case) with cubic computational complexity, assuming the 1RSB ansatz is valid.

Vector Approximate Survey Propagation

TL;DR

Simulations reveal that VASP significantly surpasses VAMP and GASP in estimation accuracy, particularly when the assumed prior is discrete-supported and the measurement matrix is non-i.i.d., and can efficiently approximate the postulated MAP estimator with only cubic complexity.

Abstract

Approximate Message Passing (AMP), originally designed to solve high-dimensional linear inverse problems, has found broad applications in signal processing and statistical inference. Among its key variants, Vector Approximate Message Passing (VAMP) and Generalized Approximate Survey Propagation (GASP) have demonstrated effectiveness even in scenarios where the assumed generative models differ from the true models. However, the maximum a posteriori (MAP) versions of VAMP and GASP have limitations: VAMP is restricted to differentiable priors and likelihoods, while GASP requires the measurement matrix to have independent identically distributed (i.i.d.) elements. To overcome these limitations, this paper introduces a new algorithm, Vector Approximate Survey Propagation (VASP). VASP utilizes survey propagation to handle non-differentiable priors and likelihoods, along with employs vector-form messages to account for correlations in the measurement matrix. Simulations reveal that VASP significantly surpasses VAMP and GASP in estimation accuracy, particularly when the assumed prior is discrete-supported and the measurement matrix is non-i.i.d.. Additionally, the state evolution (SE) of VASP, derived heuristically, accurately reflects the per-iteration mean squared error (MSE). A comparison between the SE and the free energy computed by Takahashi and Kabashima under the one-step replica symmetry breaking (1RSB) ansatz shows that the SE's fixed-point equations align with the free energy's saddle point equations. This suggests that VASP efficiently implements the postulated MAP estimator (which is NP-hard in the worst case) with cubic computational complexity, assuming the 1RSB ansatz is valid.
Paper Structure (16 sections, 3 theorems, 37 equations, 7 figures, 2 tables, 2 algorithms)

This paper contains 16 sections, 3 theorems, 37 equations, 7 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Reformulation of Kabashima's Result
  3. Revisit to Kabashima's Replica Prediction
  4. Reformulation of the 1RSB Saddle Point Equations
  5. The VASP Algorithm
  6. Analysis of Reformulated Result
  7. Definition of Vector Survey
  8. Propagation of Vector Survey
  9. The Proposed Algorithm
  10. State Evolution of the VASP
  11. SE and Its Fixed Point Equations
  12. A Heuristic Derivation for the SE
  13. Simulation Results
  14. Conclusion
  15. Fixed Point Equations of SE
  16. ...and 1 more sections

Key Result

Proposition 1

The 1RSB saddle point equations derived by Takahashi and Kabashima takahashi2022macroscopic are mathematically equivalent We do not claim the rigorousness of Kabashima's results, as they were obtained from the replica method. to the formulas provided in Tab. Tab:Saddle_Point in Appendix Appendix:Sum

Figures (7)

  • Figure 1: Replicated factor graph for the inference problem
  • Figure 2: Validation of Assumption \ref{['assump:EmpiricalConverge']} (in a QQ-plot, if two distributions compared are similar, their samples will approximately lie on the diagonal line).
  • Figure 3: Comparison in model-matched cases ($c = 0$)
  • Figure 4: Comparison in model-mismatched cases ($c = 0.01$)
  • Figure 5: Influence of matrix correlation ($v_{\mathsf{T}} = v_{\mathsf{F}} = 0.01$)
  • ...and 2 more figures

Theorems & Definitions (3)

  • Proposition 1
  • Proposition 2
  • Lemma 1