Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unifying AMP Algorithms for Rotationally-Invariant Models

Songbin Liu, Junjie Ma

TL;DR

This paper develops a unified framework for constructing AMP algorithms tailored to rotationally-invariant models by reductions to long-memory OAMP and a recursive orthogonal-centering approach. It derives the RI-AMP and its Onsager terms via free cumulants of the spectral law, and exposes two novel RI-AMP variants: RI-AMP-DF and RI-AMP-MP, with connections to BAMP and GFOM. The framework is applied to spiked models, yielding a flexible, state-evolution-aware methodology that extends beyond GOE assumptions. The work offers practical estimators for free cumulants, a clear path from FOMs to OAMP, and broad generalizations that enhance high-dimensional estimation in rotationally-invariant settings.

Abstract

This paper presents a unified framework for constructing Approximate Message Passing (AMP) algorithms for rotationally-invariant models. By employing a general iterative algorithm template and reducing it to long-memory Orthogonal AMP (OAMP), we systematically derive the correct Onsager terms of AMP algorithms. This approach allows us to rederive an AMP algorithm introduced by Fan and Opper et al., while shedding new light on the role of free cumulants of the spectral law. The free cumulants arise naturally from a recursive centering operation, potentially of independent interest beyond the scope of AMP. To illustrate the flexibility of our framework, we introduce two novel AMP variants and apply them to estimation in spiked models.

Unifying AMP Algorithms for Rotationally-Invariant Models

TL;DR

This paper develops a unified framework for constructing AMP algorithms tailored to rotationally-invariant models by reductions to long-memory OAMP and a recursive orthogonal-centering approach. It derives the RI-AMP and its Onsager terms via free cumulants of the spectral law, and exposes two novel RI-AMP variants: RI-AMP-DF and RI-AMP-MP, with connections to BAMP and GFOM. The framework is applied to spiked models, yielding a flexible, state-evolution-aware methodology that extends beyond GOE assumptions. The work offers practical estimators for free cumulants, a clear path from FOMs to OAMP, and broad generalizations that enhance high-dimensional estimation in rotationally-invariant settings.

Abstract

This paper presents a unified framework for constructing Approximate Message Passing (AMP) algorithms for rotationally-invariant models. By employing a general iterative algorithm template and reducing it to long-memory Orthogonal AMP (OAMP), we systematically derive the correct Onsager terms of AMP algorithms. This approach allows us to rederive an AMP algorithm introduced by Fan and Opper et al., while shedding new light on the role of free cumulants of the spectral law. The free cumulants arise naturally from a recursive centering operation, potentially of independent interest beyond the scope of AMP. To illustrate the flexibility of our framework, we introduce two novel AMP variants and apply them to estimation in spiked models.

Paper Structure

This paper contains 48 sections, 16 theorems, 172 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. AMP Algorithm
  3. Orthogonal AMP Algorithm
  4. Contributions
  5. Organization and Notations
  6. Organization.
  7. Notation.
  8. Preliminaries
  9. Free Cumulant
  10. Non-crossing partition.
  11. Free cumulant.
  12. A Recursive Characterization of Free Cumulants
  13. Monte Carlo Estimator of Free Cumulants
  14. Orthogonal AMP Algorithm
  15. Main Results
  16. ...and 33 more sections

Key Result

Proposition 1

Assume that the moments $(m_n)_{n\ge1}$ of a random variable $\mathsf{\Lambda}$ exist for all orders. Let $(\kappa_n)_{n\ge1}$ be the free cumulants of $\mathsf{\Lambda}$. Define a sequence of random variables $(Q_n)_{n\ge0}$ recursively as follows: where $Q_0:=1$. Then, we have

Figures (3)

  • Figure 1: A non-crossing partition of $\{1,2,3,4,5\}$: $(\{1\}, \{2,5\},\{3,4\})$.
  • Figure 2: MSE performance of the RI-AMP-MP algorithm and the BAMP algorithm barbier2023fundamental. In both experiments, we initialize the algorithms by $\bm{u}_1 = \sqrt{\omega} \bm{x}_\star + \sqrt{(1-\omega)} \bm{n}$ where $\bm{n}$ is standard Gaussian independent of $\bm{x}_\star$ and $\omega = 0.3$. The empirical results are average over 50 independent runs. $N=5000$. The SNR parameter is $\theta=3$ for figure (a) and $\theta=1.5$ for figure (b).
  • Figure 3: Left: Map from $(s_1,\ldots,s_5)=(1,0,0,2,2)\in\mathcal{S}(5)$ to a Dyck path of length 10 marked in solid lines. Right: Map from the Dyck path to a non-crossing partition of $[5]$: $(\{1\}, \{2,5\},\{3,4\})$. (The non-crossing partition is depicted in Fig. \ref{['Fig:Dyck']}.) The 5 horizontal/vertical steps of the Dyck path are marked in red/blue colors.

Theorems & Definitions (51)

  • Proposition 1
  • proof
  • Remark 1: Connection with the partial moments in fan2022approximate
  • Corollary 1
  • Remark 2: Alternative calculations of free cumulants
  • Definition 1: Orthogonal AMP algorithm
  • Definition 2: Convergence of high-dimensional vectors
  • Theorem 1: State evolution of OAMP dudeja2022spectral
  • Remark 3: Interpretation of OAMP's state evolution
  • Proposition 2: Pairwise orthogonality
  • ...and 41 more