Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Quadratic Systems with Full-Rank Matrices Using Sparse or Generative Priors

Junren Chen, Michael K. Ng, Zhaoqiang Liu

TL;DR

This paper addresses the high-dimensional case where the projected gradient descent (PGD) algorithm is developed and introduces the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level.

Abstract

The problem of recovering a signal $\boldsymbol x\in \mathbb{R}^n$ from a quadratic system $\{y_i=\boldsymbol x^\top\boldsymbol A_i\boldsymbol x,\ i=1,\ldots,m\}$ with full-rank matrices $\boldsymbol A_i$ frequently arises in applications such as unassigned distance geometry and sub-wavelength imaging. With i.i.d. standard Gaussian matrices $\boldsymbol A_i$, this paper addresses the high-dimensional case where $m\ll n$ by incorporating prior knowledge of $\boldsymbol x$. First, we consider a $k$-sparse $\boldsymbol x$ and introduce the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level $k$. TWF comprises two steps: the spectral initialization that identifies a point sufficiently close to $\boldsymbol x$ (up to a sign flip) when $m=O(k^2\log n)$, and the thresholded gradient descent which, when provided a good initialization, produces a sequence linearly converging to $\boldsymbol x$ with $m=O(k\log n)$ measurements. Second, we explore the generative prior, assuming that $x$ lies in the range of an $L$-Lipschitz continuous generative model with $k$-dimensional inputs in an $\ell_2$-ball of radius $r$. With an estimate correlated with the signal, we develop the projected gradient descent (PGD) algorithm that also comprises two steps: the projected power method that provides an initial vector with $O\big(\sqrt{\frac{k \log L}{m}}\big)$ $\ell_2$-error given $m=O(k\log(Lnr))$ measurements, and the projected gradient descent that refines the $\ell_2$-error to $O(δ)$ at a geometric rate when $m=O(k\log\frac{Lrn}{δ^2})$. Experimental results corroborate our theoretical findings and show that: (i) our approach for the sparse case notably outperforms the existing provable algorithm sparse power factorization; (ii) leveraging the generative prior allows for precise image recovery in the MNIST dataset from a small number of quadratic measurements.

Solving Quadratic Systems with Full-Rank Matrices Using Sparse or Generative Priors

TL;DR

This paper addresses the high-dimensional case where the projected gradient descent (PGD) algorithm is developed and introduces the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level.

Abstract

The problem of recovering a signal from a quadratic system with full-rank matrices frequently arises in applications such as unassigned distance geometry and sub-wavelength imaging. With i.i.d. standard Gaussian matrices , this paper addresses the high-dimensional case where by incorporating prior knowledge of . First, we consider a -sparse and introduce the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level . TWF comprises two steps: the spectral initialization that identifies a point sufficiently close to (up to a sign flip) when , and the thresholded gradient descent which, when provided a good initialization, produces a sequence linearly converging to with measurements. Second, we explore the generative prior, assuming that lies in the range of an -Lipschitz continuous generative model with -dimensional inputs in an -ball of radius . With an estimate correlated with the signal, we develop the projected gradient descent (PGD) algorithm that also comprises two steps: the projected power method that provides an initial vector with -error given measurements, and the projected gradient descent that refines the -error to at a geometric rate when . Experimental results corroborate our theoretical findings and show that: (i) our approach for the sparse case notably outperforms the existing provable algorithm sparse power factorization; (ii) leveraging the generative prior allows for precise image recovery in the MNIST dataset from a small number of quadratic measurements.
Paper Structure (41 sections, 19 theorems, 122 equations, 8 figures, 4 algorithms)

This paper contains 41 sections, 19 theorems, 122 equations, 8 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Problem Set-up
  3. Prior Art
  4. The Rank-1 Model --- (Sparse) Phase Retrieval
  5. The Full-Rank Quadratic System
  6. Generative Priors for Quadratic Systems
  7. Our Contribution
  8. Sparse Quadratic System
  9. Generative Quadratic System
  10. Notations and Preliminaries
  11. Notations
  12. Preliminaries
  13. Solving Sparse Quadratic System via Thresholded Wirtinger Flow
  14. Algorithm: Initialization and Refinement
  15. Theory: Linear Convergence Guarantee
  16. ...and 26 more sections

Key Result

lemma 1

(Bernstein's inequality, vershynin2018high) Let $X_1,...,X_N$ be independent, zero-mean, sub-exponential random variables, then for every $t\geq 0$ and some absolute constant $c$, we let $A=\sum_{i=1}^N\|X_i\|^2_{\psi_2}$, $B=\max_{1\leq i\leq N}\|X_i\|_{\psi_1}$ and have

Figures (8)

  • Figure 1: Comparison of the standard spectral initialization (SI) and the proposed spectral initialization with estimated sparsity support (SI-S). The relative distances from the initializers to a global optimizer were calculated with $k=5$, $n=500$ and varying number of measurements $m$.
  • Figure 2: The success rates of WF, SPF and TWF obtained by varying the sparsity level $k$ and the number of measurements $m$.
  • Figure 3: TWF achieves stable recovery of $\bm{x}\in\mathbb{S}^{n-1}$ under additive Gaussian noise.
  • Figure 4: Noiseless reconstructions for MNIST images with $m =120$.
  • Figure 5: Noiseless reconstructions for MNIST images with $m =400$.
  • ...and 3 more figures

Theorems & Definitions (36)

  • lemma 1
  • lemma 2
  • Proposition 1
  • proof
  • Remark 1
  • lemma 3
  • proof
  • lemma 4
  • proof
  • lemma 5
  • ...and 26 more