Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compressed Sensing with Upscaled Vector Approximate Message Passing

Nikolajs Skuratovs, Michael Davies

TL;DR

This work considers the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator and proposes a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction.

Abstract

The Recently proposed Vector Approximate Message Passing (VAMP) algorithm demonstrates a great reconstruction potential at solving compressed sensing related linear inverse problems. VAMP provides high per-iteration improvement, can utilize powerful denoisers like BM3D, has rigorously defined dynamics and is able to recover signals measured by highly undersampled and ill-conditioned linear operators. Yet, its applicability is limited to relatively small problem sizes due to the necessity to compute the expensive LMMSE estimator at each iteration. In this work we consider the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator. We propose a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction. To further improve the performance of CG-VAMP, we design a warm-starting scheme for CG and develop theoretical models for the Onsager correction and the State Evolution of Warm-Started CG-VAMP (WS-CG-VAMP). Additionally, we develop robust and accurate methods for implementing the WS-CG-VAMP algorithm. The numerical experiments on large-scale image reconstruction problems demonstrate that WS-CG-VAMP requires much fewer CG iterations compared to CG-VAMP to achieve the same or superior level of reconstruction.

Compressed Sensing with Upscaled Vector Approximate Message Passing

TL;DR

This work considers the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator and proposes a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction.

Abstract

The Recently proposed Vector Approximate Message Passing (VAMP) algorithm demonstrates a great reconstruction potential at solving compressed sensing related linear inverse problems. VAMP provides high per-iteration improvement, can utilize powerful denoisers like BM3D, has rigorously defined dynamics and is able to recover signals measured by highly undersampled and ill-conditioned linear operators. Yet, its applicability is limited to relatively small problem sizes due to the necessity to compute the expensive LMMSE estimator at each iteration. In this work we consider the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator. We propose a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction. To further improve the performance of CG-VAMP, we design a warm-starting scheme for CG and develop theoretical models for the Onsager correction and the State Evolution of Warm-Started CG-VAMP (WS-CG-VAMP). Additionally, we develop robust and accurate methods for implementing the WS-CG-VAMP algorithm. The numerical experiments on large-scale image reconstruction problems demonstrate that WS-CG-VAMP requires much fewer CG iterations compared to CG-VAMP to achieve the same or superior level of reconstruction.

Paper Structure

This paper contains 23 sections, 9 theorems, 132 equations, 6 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notations
  3. Expectation Propagation framework and its extensions
  4. Background on Expectation Propagation
  5. The Large System Limit model of EP
  6. Approximate EP -- Vector Approximate Message Passing
  7. Conjugate Gradient for upscaling VAMP
  8. Large System Limit dynamics of CG-VAMP
  9. Stable implementation of CG-VAMP
  10. Practical estimation of $v_{A \rightarrow B}^{t,i}$ in CG-VAMP
  11. Adaptive CG in CG-VAMP
  12. Warm-starting the CG algorithm in CG-VAMP
  13. Rigorous dynamics of Block A in WS-CG-VAMP
  14. Robust estimation of Block A parameters in WS-CG-VAMP
  15. Approximate update $\mathbf{x}_{A \rightarrow B}^t$ in WS-CG-VAMP
  16. ...and 8 more sections

Key Result

Lemma 1

VAMP, EP_Keigo: Let $\mathbf{g}_A$ be the LMMSE estimator eq:LMMSE and $\mathbf{g}_B$ be the MMSE denoiser. Then, under Assumptions 1-3, the variances $v_{B \rightarrow A}^t \overset{a.s.}{=} \lim_{N \rightarrow \infty} \frac{1}{N} || \mathbf{x}_{B \rightarrow A}^t - \mathbf{x} ||^2$ and $v_{A \righ

Figures (6)

  • Figure 1: The ground truth image to be reconstructed.
  • Figure 2: NMSE versus outer-loop iteration number $t$. The dashed curves correspond to CG-VAMP A, while the solid lines represent the proposed CG-VAMP B.
  • Figure 3: The number of CG iterations $i[t]$ that follow only the stopping criterion \ref{['eq:CG_stopping_rule']} (red curve) and that follows both \ref{['eq:CG_stopping_rule']} and \ref{['eq:CG_stopping_rule_2']} (blue curve).
  • Figure 4: The NMSE and the computational time of CG-VAMP with ACG with two types of stopping criteria.
  • Figure 5: The NMSE of VAMP, CG-VAMP B and different versions of WS-CG-VAMP versus outer-loop iteration $t$.
  • ...and 1 more figures

Theorems & Definitions (22)

  • Lemma 1
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof
  • Theorem 5
  • ...and 12 more