Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal convex lifted sparse phase retrieval and PCA with an atomic matrix norm regularizer

Andrew D. McRae, Justin Romberg, Mark A. Davenport

Abstract

We present novel analysis and algorithms for solving sparse phase retrieval and sparse principal component analysis (PCA) with convex lifted matrix formulations. The key innovation is a new mixed atomic matrix norm that, when used as regularization, promotes low-rank matrices with sparse factors. We show that convex programs with this atomic norm as a regularizer provide near-optimal sample complexity and error rate guarantees for sparse phase retrieval and sparse PCA. While we do not know how to solve the convex programs exactly with an efficient algorithm, for the phase retrieval case we carefully analyze the program and its dual and thereby derive a practical heuristic algorithm. We show empirically that this practical algorithm performs similarly to existing state-of-the-art algorithms.

Optimal convex lifted sparse phase retrieval and PCA with an atomic matrix norm regularizer

Abstract

We present novel analysis and algorithms for solving sparse phase retrieval and sparse principal component analysis (PCA) with convex lifted matrix formulations. The key innovation is a new mixed atomic matrix norm that, when used as regularization, promotes low-rank matrices with sparse factors. We show that convex programs with this atomic norm as a regularizer provide near-optimal sample complexity and error rate guarantees for sparse phase retrieval and sparse PCA. While we do not know how to solve the convex programs exactly with an efficient algorithm, for the phase retrieval case we carefully analyze the program and its dual and thereby derive a practical heuristic algorithm. We show empirically that this practical algorithm performs similarly to existing state-of-the-art algorithms.

Paper Structure

This paper contains 24 sections, 11 theorems, 124 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Sparsity, phase retrieval, and PCA
  3. Sparse phase retrieval
  4. Sparse PCA
  5. Key tool: A sparsity-and-low-rank--inducing atomic norm
  6. Theoretical guarantees for atomic-norm regularized estimators
  7. Sparse phase retrieval
  8. Sparse PCA
  9. PSD constraints and another regularizer
  10. Proof highlights
  11. Sparse phase retrieval proof
  12. Sparse PCA proof sketch
  13. Computational limitations and a practical algorithm for phase retrieval
  14. Factorization, duality, and optimality conditions
  15. A first factored algorithm, a computational snag, and a heuristic
  16. ...and 9 more sections

Key Result

Theorem \oldthetheorem

Suppose assump:spr_measassump:spr_noise hold. Suppose $\beta^*$ is $s$-sparse and that the number of measurements $n$ satisfies $n \gtrsim s \log (ep/s)$. If the regularization parameter satisfies where $c \approx (s \log (ep/s))^{-1}$, then, with probability at least $1 - e^{-bn} - e^{-s} (s/p)^s$ (where $b > 0$ is a constant), the estimator $\Bhat$ from eq:pr_opt_main satisfies

Figures (2)

  • Figure 1: Phase transition plots. Colors represent 80% quantile error over 20 trials (darker colors correspond to higher error). We used $p = 20{,}000$, $\lVert\beta^*\rVert_2 = 1$, and $\sigma = 0.05$. All algorithms were run on the same data.
  • Figure 2: Plot of $\lVert\betahat - \beta^*\rVert_2$ vs. $s$ (80% quantile over 10 trials). All simulations use $p = 8{,}000$ and $n = 4{,}000$. Blue circles are actual data; the red curves are of the form $c \sqrt{s \log \frac{ep}{s}}$, where the scaling factor $c$ is chosen to give minimum mean absolute deviation.

Theorems & Definitions (27)

  • Theorem \oldthetheorem
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6
  • Theorem \oldthetheorem
  • Remark 7
  • Remark 8
  • ...and 17 more