Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast, robust approximate message passing

Misha Ivkov, Tselil Schramm

Abstract

We give a fast, spectral procedure for implementing approximate-message passing (AMP) algorithms robustly. For any quadratic optimization problem over symmetric matrices $X$ with independent subgaussian entries, and any separable AMP algorithm $\mathcal A$, our algorithm performs a spectral pre-processing step and then mildly modifies the iterates of $\mathcal A$. If given the perturbed input $X + E \in \mathbb R^{n \times n}$ for any $E$ supported on a $\varepsilon n \times \varepsilon n$ principal minor, our algorithm outputs a solution $\hat v$ which is guaranteed to be close to the output of $\mathcal A$ on the uncorrupted $X$, with $\|\mathcal A(X) - \hat v\|_2 \le f(\varepsilon) \|\mathcal A(X)\|_2$ where $f(\varepsilon) \to 0$ as $\varepsilon \to 0$ depending only on $\varepsilon$.

Fast, robust approximate message passing

Abstract

We give a fast, spectral procedure for implementing approximate-message passing (AMP) algorithms robustly. For any quadratic optimization problem over symmetric matrices with independent subgaussian entries, and any separable AMP algorithm , our algorithm performs a spectral pre-processing step and then mildly modifies the iterates of . If given the perturbed input for any supported on a principal minor, our algorithm outputs a solution which is guaranteed to be close to the output of on the uncorrupted , with where as depending only on .

Paper Structure

This paper contains 12 sections, 16 theorems, 55 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Setup and definitions
  3. Results
  4. Experiments
  5. Discussion
  6. Technical overview
  7. AMP preliminaries
  8. Making AMP robust to principal minor corruptions
  9. Spectral cleaning
  10. Analysis of clipped AMP on spectrally cleaned input
  11. AMP is robust to small spectral perturbations
  12. Statistics of AMP iterate entries

Key Result

Theorem 1.4

Suppose $\mathcal{A}$ is a $T$-step AMP algorithm with $O(1)$-Lipschitz or polynomial denoiser functions. Let $X$ be a symmetric $n \times n$ matrix with i.i.d. $\frac{O(1)}{\sqrt{n}}$-subgaussian entries having mean $0$ and variance $\frac{1}{n}$, and let $v_{\mathrm{AMP}}(X)$ be the output of $\ma with probability $1-o(1)$ over the randomness of $X$, where $d = 1$ if the denoisers are Lipschitz,

Figures (1)

  • Figure 1: Plot of the correlation of the vector $\hat{v}(Y)$ with the output of AMP on the "clean" matrix $X$, and of the objective value attained by $\hat{v}(Y)$ on the clean matrix $X$.

Theorems & Definitions (36)

  • Definition 1.1: AMP algorithm
  • Example 1.2: non-negative PCA
  • Definition 1.3: $\varepsilon$-principal minor corruption
  • Theorem 1.4: Informal version of thm:main-principal
  • Corollary 1.5: Fast, robust Sherrington Kirkpatrick
  • Definition 2.1: Onsager correction
  • Definition 2.2: Pseudo-Lipschitz Functions
  • Corollary 2.3
  • Theorem 3.1: Main Theorem
  • Definition 3.2
  • ...and 26 more