Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate Projections onto the Positive Semidefinite Cone Using Randomization

Morgan Jones, James Anderson

TL;DR

Two novel algorithms for approximately projecting symmetric matrices onto the Positive Semidefinite (PSD) cone using Randomized Numerical Linear Algebra (RNLA) offer a trade-off between accuracy and computational speed, supported by probabilistic error bounds.

Abstract

This paper presents two novel algorithms for approximately projecting symmetric matrices onto the Positive Semidefinite (PSD) cone using Randomized Numerical Linear Algebra (RNLA). Classical PSD projection methods rely on full-rank deterministic eigen-decomposition, which can be computationally prohibitive for large-scale problems. Our approach leverages RNLA to construct low-rank matrix approximations before projection, significantly reducing the required numerical resources. The first algorithm utilizes random sampling to generate a low-rank approximation, followed by a standard eigen-decomposition on this smaller matrix. The second algorithm enhances this process by introducing a scaling approach that aligns the leading-order singular values with the positive eigenvalues, ensuring that the low-rank approximation captures the essential information about the positive eigenvalues for PSD projection. Both methods offer a trade-off between accuracy and computational speed, supported by probabilistic error bounds. To further demonstrate the practical benefits of our approach, we integrate the randomized projection methods into a first-order Semi-Definite Programming (SDP) solver. Numerical experiments, including those on SDPs derived from Sum-of-Squares (SOS) programming problems, validate the effectiveness of our method, especially for problems that are infeasible with traditional deterministic methods.

Approximate Projections onto the Positive Semidefinite Cone Using Randomization

TL;DR

Two novel algorithms for approximately projecting symmetric matrices onto the Positive Semidefinite (PSD) cone using Randomized Numerical Linear Algebra (RNLA) offer a trade-off between accuracy and computational speed, supported by probabilistic error bounds.

Abstract

This paper presents two novel algorithms for approximately projecting symmetric matrices onto the Positive Semidefinite (PSD) cone using Randomized Numerical Linear Algebra (RNLA). Classical PSD projection methods rely on full-rank deterministic eigen-decomposition, which can be computationally prohibitive for large-scale problems. Our approach leverages RNLA to construct low-rank matrix approximations before projection, significantly reducing the required numerical resources. The first algorithm utilizes random sampling to generate a low-rank approximation, followed by a standard eigen-decomposition on this smaller matrix. The second algorithm enhances this process by introducing a scaling approach that aligns the leading-order singular values with the positive eigenvalues, ensuring that the low-rank approximation captures the essential information about the positive eigenvalues for PSD projection. Both methods offer a trade-off between accuracy and computational speed, supported by probabilistic error bounds. To further demonstrate the practical benefits of our approach, we integrate the randomized projection methods into a first-order Semi-Definite Programming (SDP) solver. Numerical experiments, including those on SDPs derived from Sum-of-Squares (SOS) programming problems, validate the effectiveness of our method, especially for problems that are infeasible with traditional deterministic methods.

Paper Structure

This paper contains 9 sections, 15 theorems, 57 equations, 6 figures, 2 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Projecting onto the PSD Cone
  4. Low-Rank Randomized PSD Projection
  5. A Scaled Randomized PSD Projection
  6. PSD Projection Numerical Examples
  7. Numerical Examples
  8. Conclusion
  9. Appendix

Key Result

Theorem 1

Consider $X \in \mathbb{S}_n$. Then there is a unique solution to the optimization problem given in Equation opt: projecting onto PSD given by where $X = UDU^\top$ is the eigen-decomposition of $X \in \mathbb{S}_n$.

Figures (6)

  • Figure 1: Projection descriptions and properties.
  • Figure 2: Comparing $\|A-B\|$, $\|A_+-B_+\|$ where $A=\mathop{\mathrm{diag}}[1, 100]$ and $B = 0200200d$ and $d$ is varied between $[0,200]$. Here, PSD projections are calculated using Eq. \ref{['eq: analytical PSD proj using eigenvalue']}, where the eigen-decomposition is calculated using Matlab's eig function. It is clear that $\|A_+-B_+\|_2 \le \|A-B\|_2$ but this is not the case with the spectral norm, $\|A_+-B_+\|_\infty \not \le \|A-B\|_\infty$.
  • Figure 3: Plot showing the distribution of eigenvalues and singular values of the matrix given in Eq. \ref{['eq: matrix where scale is better']} and corresponding associated scalars $\beta_3>\beta_1>\beta_4>\beta_2>0$.
  • Figure 4: Accuracy of projecting $X$ from Eq. \ref{['eq: matrix where scale is better']} with $n=1000$, $\beta_1=3$, $\beta_2=1$, $\beta_3=6$, $\beta_4=2$ and $Y$ randomly generated using Algorithms \ref{['alg:approx proj']} and \ref{['alg: scalled approx proj']} under $l=5$, $q=2$ and varying $k \in \mathbb{N}$. The true PSD projections are calculated using Eq. \ref{['eq: analytical PSD proj using eigenvalue']}, where the eigen-decomposition is calculated using Matlab's eig function.
  • Figure 5: Scatter plot showing the distribution of the eigenvalues $\lambda_1 \ge \dots \ge \lambda_n$ for various benchmark matrices from davis2011university, whose projections are analysed in Table \ref{['table: ran PSD projections']}.
  • ...and 1 more figures

Theorems & Definitions (30)

  • Theorem 1: Page 399 boyd2004convex
  • Lemma 1
  • proof
  • Proposition 1: PSD projection bounds
  • proof
  • Corollary 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • ...and 20 more