Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Markov Chains Approximate Message Passing

Amit Rajaraman, David X. Wu

TL;DR

This work investigates the spiked Wigner inference problem and asks whether polynomial-time Markov chains like Glauber dynamics can attain Bayes-optimal performance in sampling from the posterior. It introduces Restricted Gaussian Dynamics (RGD) and shows that, via a one-dimensional correlation recursion, RGD mirrors the state evolution of Approximate Message Passing (AMP), thereby tying Glauber-type dynamics to Bayes-optimal recovery in the high-SNR regime. The authors establish a precise connection between RGD and AMP, analyze the fixed-point structure of the induced one-dimensional map, and reveal a phase-transition governed by the Almeida–Thouless line and SK-model mixing conditions. Conditional on rapid SK mixing, Glauber dynamics achieve nontrivial recovery with fixed-point OPT_{β,λ}, providing insight into pre-mixing behavior and potential annealed sampling algorithms for Bayes-optimal inference in spiked matrix models.

Abstract

Markov chain Monte Carlo algorithms have long been observed to obtain near-optimal performance in various Bayesian inference settings. However, developing a supporting theory that makes these studies rigorous has proved challenging. In this paper, we study the classical spiked Wigner inference problem, where one aims to recover a planted Boolean spike from a noisy matrix measurement. We relate the recovery performance of Glauber dynamics on the annealed posterior to the performance of Approximate Message Passing (AMP), which is known to achieve Bayes-optimal performance. Our main results rely on the analysis of an auxiliary Markov chain called restricted Gaussian dynamics (RGD). Concretely, we establish the following results: 1. RGD can be reduced to an effective one-dimensional recursion which mirrors the evolution of the AMP iterates. 2. From a warm start, RGD rapidly converges to a fixed point in correlation space, which recovers Bayes-optimal performance when run on the posterior. 3. Conditioned on widely believed mixing results for the SK model, we recover the phase transition for non-trivial inference.

Markov Chains Approximate Message Passing

TL;DR

This work investigates the spiked Wigner inference problem and asks whether polynomial-time Markov chains like Glauber dynamics can attain Bayes-optimal performance in sampling from the posterior. It introduces Restricted Gaussian Dynamics (RGD) and shows that, via a one-dimensional correlation recursion, RGD mirrors the state evolution of Approximate Message Passing (AMP), thereby tying Glauber-type dynamics to Bayes-optimal recovery in the high-SNR regime. The authors establish a precise connection between RGD and AMP, analyze the fixed-point structure of the induced one-dimensional map, and reveal a phase-transition governed by the Almeida–Thouless line and SK-model mixing conditions. Conditional on rapid SK mixing, Glauber dynamics achieve nontrivial recovery with fixed-point OPT_{β,λ}, providing insight into pre-mixing behavior and potential annealed sampling algorithms for Bayes-optimal inference in spiked matrix models.

Abstract

Markov chain Monte Carlo algorithms have long been observed to obtain near-optimal performance in various Bayesian inference settings. However, developing a supporting theory that makes these studies rigorous has proved challenging. In this paper, we study the classical spiked Wigner inference problem, where one aims to recover a planted Boolean spike from a noisy matrix measurement. We relate the recovery performance of Glauber dynamics on the annealed posterior to the performance of Approximate Message Passing (AMP), which is known to achieve Bayes-optimal performance. Our main results rely on the analysis of an auxiliary Markov chain called restricted Gaussian dynamics (RGD). Concretely, we establish the following results: 1. RGD can be reduced to an effective one-dimensional recursion which mirrors the evolution of the AMP iterates. 2. From a warm start, RGD rapidly converges to a fixed point in correlation space, which recovers Bayes-optimal performance when run on the posterior. 3. Conditioned on widely believed mixing results for the SK model, we recover the phase transition for non-trivial inference.

Paper Structure

This paper contains 29 sections, 30 theorems, 120 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Beyond worst-case mixing.
  3. Main results
  4. Related work
  5. Spiked Wigner model.
  6. Bayes optimal estimation.
  7. Dynamics of Markov chains before mixing.
  8. Locally stationary distributions.
  9. Organization
  10. Technical Overview
  11. Reduction to one dimension.
  12. The mean recursion and fixed point structure
  13. RGD simulates AMP
  14. Qualitative behavior of the fixed point equations
  15. Conjectures and open questions
  16. ...and 14 more sections

Key Result

Theorem 1.5

Let $\boldsymbol{\sigma}_0 \in \{ \pm 1 \}^N$ be arbitrary, and let $(\boldsymbol{\sigma}_t)_{t \geqslant 0}$ be the trajectory of the Glauber dynamics run with stationary distribution $\mu_{\beta\boldsymbol{M}}$ initialized at $\boldsymbol{\sigma}_0$. Let $T \geqslant \widetilde{\Omega}(N^{4})$, an for some constant $\mathsf{OPT}_{\beta,\lambda}$ that is the largest fixed point of an explicit AMP

Figures (2)

  • Figure 1: An illustration of our results.
  • Figure 2: Behavior of the RGD recursion for different values of $\beta$.

Theorems & Definitions (73)

  • Definition 1.3: Restricted Gaussian dynamics STL20LST21CE22
  • Definition 1.4
  • Theorem 1.5: Informal, see \ref{['thm:glauber-formal']}
  • Remark 1.6
  • Theorem 1.7: Informal, see \ref{['subsec:rgd-amp-simulation']}
  • Remark 1.8
  • Theorem 1.9: Informal, see \ref{['th:rgd-fixed-temp']}
  • Remark 1.10
  • Remark 1.11
  • Theorem 1.12: Informal, see \ref{['lem:all-field-mag-conc']}
  • ...and 63 more