Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

K-step Vector Approximate Survey Propagation

Qun Chen, Haochuan Zhang, Huimin Zhu

TL;DR

Simulations show that KVASP significantly outperforms VAMP and GASP in estimation accuracy, particularly when the assumed prior has discrete support and the measurement matrix is non-i.i.d..

Abstract

Approximate Message Passing (AMP), originally developed to address high-dimensional linear inverse problems, has found widespread applications in signal processing and statistical inference. Among its notable variants, Vector Approximate Message Passing (VAMP), Generalized Approximate Survey Propagation (GASP), and Vector Approximate Survey Propagation (VASP) have demonstrated effectiveness even when the assumed generative models differ from the true models. However, many fundamental questions regarding model mismatch remain unanswered. For instance, it is still unclear what level of model mismatch is required for the postulated posterior estimate (PPE) to exhibit a replica symmetry breaking (RSB) structure in the extremum conditions of its free energy, and what order of RSB is necessary. In this paper, we introduce a novel approximate message passing algorithm that incorporates K-step RSB (KRSB) and naturally reduces to VAMP and VASP with specific parameter selections. We refer to this as the K-step VASP (KVASP) algorithm. Simulations show that KVASP significantly outperforms VAMP and GASP in estimation accuracy, particularly when the assumed prior has discrete support and the measurement matrix is non-i.i.d.. Additionally, the state evolution (SE) of KVASP, derived heuristically, accurately tracks the per-iteration mean squared error (MSE). A comparison between the SE and the free energy under the KRSB ansatz reveals that the fixed-point equations of SE align with the saddle-point equations of the free energy. This suggests that, once the KRSB ansatz holds and the SE fixed point is reached, KVASP can accurately compute the PPE in the large system limit (LSL).

K-step Vector Approximate Survey Propagation

TL;DR

Simulations show that KVASP significantly outperforms VAMP and GASP in estimation accuracy, particularly when the assumed prior has discrete support and the measurement matrix is non-i.i.d..

Abstract

Approximate Message Passing (AMP), originally developed to address high-dimensional linear inverse problems, has found widespread applications in signal processing and statistical inference. Among its notable variants, Vector Approximate Message Passing (VAMP), Generalized Approximate Survey Propagation (GASP), and Vector Approximate Survey Propagation (VASP) have demonstrated effectiveness even when the assumed generative models differ from the true models. However, many fundamental questions regarding model mismatch remain unanswered. For instance, it is still unclear what level of model mismatch is required for the postulated posterior estimate (PPE) to exhibit a replica symmetry breaking (RSB) structure in the extremum conditions of its free energy, and what order of RSB is necessary. In this paper, we introduce a novel approximate message passing algorithm that incorporates K-step RSB (KRSB) and naturally reduces to VAMP and VASP with specific parameter selections. We refer to this as the K-step VASP (KVASP) algorithm. Simulations show that KVASP significantly outperforms VAMP and GASP in estimation accuracy, particularly when the assumed prior has discrete support and the measurement matrix is non-i.i.d.. Additionally, the state evolution (SE) of KVASP, derived heuristically, accurately tracks the per-iteration mean squared error (MSE). A comparison between the SE and the free energy under the KRSB ansatz reveals that the fixed-point equations of SE align with the saddle-point equations of the free energy. This suggests that, once the KRSB ansatz holds and the SE fixed point is reached, KVASP can accurately compute the PPE in the large system limit (LSL).

Paper Structure

This paper contains 12 sections, 2 theorems, 16 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Derivation of this KRSB Free Energy
  3. The KVASP Algorithm
  4. Definition of the Generalized Vector Survey
  5. Propagation of the Generalized Vector Survey
  6. State Evolution of the KVASP
  7. SE and Its Fixed Point Equations
  8. Simulation Results
  9. Conclusion
  10. Fixed Point Equations of the SE
  11. Equations for MIMO Detection Problem
  12. A Useful Lemma

Key Result

Proposition 1

The fixed point equations of the KVASP's SE align with Algo. Tab:Saddle_Point.

Figures (3)

  • Figure 1: Replicated factor graph for the inference problem
  • Figure 2: Comparison in model-mismatched cases ($K = 1$ and $L_{1} = 4$)
  • Figure 3: Comparison in model-mismatched cases ($K = 2$, $L_{1} = 2$ and $L_{2} = 4$)

Theorems & Definitions (2)

  • Proposition 1
  • Lemma 1