Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derivation of the AMP equations from belief propagation for the $\ell_2$ minimisation problem

Giuseppe Genovese, Arianna Piana

TL;DR

The work provides a fully rigorous bridge from belief propagation to the approximate message passing framework for the ℓ2 minimization with a random, sub-Gaussian design. By leveraging a Gaussian initialization and a careful CLT/Edgeworth analysis, the authors show that BP marginals’ means follow AMP recursions while variances remain edge-independent, enabling a perturbative derivation of AMP from BP beyond tree-structured graphs. They prove that, in the large-N limit and with sufficiently large β, the BP-derived AMP iterates converge to the true ℓ2 minimizer x^* = A^T(AA^T)^{−1}y, thereby validating AMP as an accurate summary of BP in this setting. The results provide a robust theoretical link between BP and AMP, with potential implications for understanding BP-AMP relationships in high-dimensional estimation problems and a methodological framework for extending to more general models via Gaussian-approximation and shadowing techniques.

Abstract

We consider the $\ell_p$-minimisation, which consists of finding the vector $x\in\mathbb{R}^N$ which minimises $\|x\|_p$ subject to the linear constraint $y=Ax$, where $y\in\mathbb{R}^m$ is given and $A$ is a $m\times N$ random matrix with i.i.d. sub-Gaussian centred entries ($m<N$). This can be viewed as the zero temperature version of a statistical mechanics problem, in which one introduces a suitable Gibbs measure on $\mathbb{R}^N$. To such a Gibbs measure there are associated belief propagation equations. We prove in the easiest case $p=2$ that the means of the distributions obtained by the belief propagation iteration satisfy asymptotically the approximate message passing equations.

Derivation of the AMP equations from belief propagation for the $\ell_2$ minimisation problem

TL;DR

The work provides a fully rigorous bridge from belief propagation to the approximate message passing framework for the ℓ2 minimization with a random, sub-Gaussian design. By leveraging a Gaussian initialization and a careful CLT/Edgeworth analysis, the authors show that BP marginals’ means follow AMP recursions while variances remain edge-independent, enabling a perturbative derivation of AMP from BP beyond tree-structured graphs. They prove that, in the large-N limit and with sufficiently large β, the BP-derived AMP iterates converge to the true ℓ2 minimizer x^* = A^T(AA^T)^{−1}y, thereby validating AMP as an accurate summary of BP in this setting. The results provide a robust theoretical link between BP and AMP, with potential implications for understanding BP-AMP relationships in high-dimensional estimation problems and a methodological framework for extending to more general models via Gaussian-approximation and shadowing techniques.

Abstract

We consider the -minimisation, which consists of finding the vector which minimises subject to the linear constraint , where is given and is a random matrix with i.i.d. sub-Gaussian centred entries (). This can be viewed as the zero temperature version of a statistical mechanics problem, in which one introduces a suitable Gibbs measure on . To such a Gibbs measure there are associated belief propagation equations. We prove in the easiest case that the means of the distributions obtained by the belief propagation iteration satisfy asymptotically the approximate message passing equations.
Paper Structure (18 sections, 44 theorems, 375 equations, 1 figure)

This paper contains 18 sections, 44 theorems, 375 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Set up
  3. Main Result
  4. Oveview of the proof
  5. Outline of the paper
  6. Notations
  7. Acknowledgments.
  8. Exact computations with a Gaussian initial distribution I: MP equations
  9. Tail control of the messages
  10. Exact computations with a Gaussian initial distribution II: AMP equations
  11. Gaussian approximations for $t=1$
  12. Successive Gaussian approximations
  13. Two shadowing sequences
  14. Analysis of the AMP equations
  15. Equivalence of $L^2$ norms of random tensors
  16. ...and 3 more sections

Key Result

Theorem 1.1

Let $m<N\in\mathbb{N}$ and $\delta:=m/N$. Let $A\in \mathbb{R}^{m\times N}$ be a random matrix with i.i.d. sub-Gaussian symmetric entries with $\mathbb E[A_{12}^2]=1/m$, $\mathbb P(A_{12}=0)=0$ and satisfying the condition (eq:hiper-intro). Let also $y\in\mathbb{R}^m$ with $\max_{b\in[m]}|y_b|\;\leq Recall eq:xandv. Then, for all $(i,a)\in[N]\times[m]$ we have Moreover, let $\Delta^{(t)} \coloneq

Figures (1)

  • Figure 1: Complete bipartite factor graph for the BP iteration. The squares are the factor nodes, the dots are variable nodes.

Theorems & Definitions (88)

  • Theorem 1.1
  • proof
  • Corollary 1.2
  • Proposition 2.1
  • Lemma 2.2
  • proof
  • Lemma 2.3
  • proof
  • Lemma 2.4
  • proof
  • ...and 78 more