Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Projections onto Spectral Matrix Cones

Daniel Cederberg, Stephen Boyd

TL;DR

This work introduces spectral matrix cones as a new tool for first-order conic solvers handling semidefinite programs. The core idea is that projecting onto a spectral matrix cone K_F can be performed by projecting the eigenvalues (or singular values) onto an associated spectral vector cone K_f, drastically reducing the per-iteration cost to the eigen/ singular-value step plus a cheaper projection. The authors integrate these projections into the Splitting Conic Solver (SCS) and demonstrate substantial speedups—often up to an order of magnitude—in SDPs arising from experimental design, robust PCA, and graph partitioning, with overhead from the eigen-decomposition remaining dominant. They further extend the theory to unitarily invariant functions, derive dual cones, and develop both ad-hoc and systematic projection strategies that exploit Newton and interior-point methods. Overall, spectral matrix cones offer a principled way to obtain smaller, simpler conic formulations and accelerate large-scale SDP solving in practice.

Abstract

Semidefinite programming is a fundamental problem class in convex optimization, but despite recent advances in solvers, solving large-scale semidefinite programs remains challenging. Generally the matrix functions involved are spectral or unitarily invariant, i.e., they depend only on the eigenvalues or singular values of the matrix. This paper investigates how spectral matrix cones -- cones defined from epigraphs and perspectives of spectral or unitarily invariant functions -- can be used to enhance first-order conic solvers for semidefinite programs. Our main result shows that projecting a matrix can be reduced to projecting its eigenvalues or singular values, which we demonstrate can be done at a negligible cost compared to the eigenvalue or singular value decomposition itself. We have integrated support for spectral matrix cone projections into the Splitting Conic Solver (SCS). Numerical experiments show that SCS with this enhancement can achieve speedups of up to an order of magnitude for solving semidefinite programs arising in experimental design, robust principal component analysis, and graph partitioning.

Projections onto Spectral Matrix Cones

TL;DR

This work introduces spectral matrix cones as a new tool for first-order conic solvers handling semidefinite programs. The core idea is that projecting onto a spectral matrix cone K_F can be performed by projecting the eigenvalues (or singular values) onto an associated spectral vector cone K_f, drastically reducing the per-iteration cost to the eigen/ singular-value step plus a cheaper projection. The authors integrate these projections into the Splitting Conic Solver (SCS) and demonstrate substantial speedups—often up to an order of magnitude—in SDPs arising from experimental design, robust PCA, and graph partitioning, with overhead from the eigen-decomposition remaining dominant. They further extend the theory to unitarily invariant functions, derive dual cones, and develop both ad-hoc and systematic projection strategies that exploit Newton and interior-point methods. Overall, spectral matrix cones offer a principled way to obtain smaller, simpler conic formulations and accelerate large-scale SDP solving in practice.

Abstract

Semidefinite programming is a fundamental problem class in convex optimization, but despite recent advances in solvers, solving large-scale semidefinite programs remains challenging. Generally the matrix functions involved are spectral or unitarily invariant, i.e., they depend only on the eigenvalues or singular values of the matrix. This paper investigates how spectral matrix cones -- cones defined from epigraphs and perspectives of spectral or unitarily invariant functions -- can be used to enhance first-order conic solvers for semidefinite programs. Our main result shows that projecting a matrix can be reduced to projecting its eigenvalues or singular values, which we demonstrate can be done at a negligible cost compared to the eigenvalue or singular value decomposition itself. We have integrated support for spectral matrix cone projections into the Splitting Conic Solver (SCS). Numerical experiments show that SCS with this enhancement can achieve speedups of up to an order of magnitude for solving semidefinite programs arising in experimental design, robust principal component analysis, and graph partitioning.

Paper Structure

This paper contains 52 sections, 2 theorems, 94 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Conic convex optimization.
  3. Spectral matrix cones.
  4. Spectral vector cones and projections onto spectral matrix cones.
  5. Consequences of efficient projections.
  6. Extension to unitarily invariant functions on ${\hbox{\bf R}}^{m \times n}$.
  7. Additional related work.
  8. Outline.
  9. Motivating examples
  10. Log-determinant cone
  11. Nuclear norm cone
  12. Spectral matrix cone projections
  13. Main results
  14. Examples of spectral cone pairs
  15. Log-determinant cone.
  16. ...and 37 more sections

Key Result

Theorem 1

Let $f:{\hbox{\bf R}}^n \to {\hbox{\bf R}} \cup \{\infty\}$ be a symmetric convex function corresponding to the spectral function $F:{\hbox{\bf S}}^n \to {\hbox{\bf R}} \cup \{\infty\}$. Consider the projection of $(\bar{t}, \bar{v}, \bar{X}) \in {\hbox{\bf R}} \times {\hbox{\bf R}} \times {\hbox{\b where $\Pi_{K_F}(\bar{t}, \bar{v}, \bar{X})_1 \in {\hbox{\bf R}}, \: \Pi_{K_F}(\bar{t}, \bar{v}, \b

Figures (5)

  • Figure 1: Results for experimental design for standard accuracy $\epsilon_{\text{abs}} = \epsilon_{\text{rel}} = 10^{-4}$ (top row) and lower accuracy $\epsilon_{\text{abs}} = \epsilon_{\text{rel}} = 10^{-3}$ (bottom row). The first column shows the total solve time, the second column shows the total number of iterations, the third column shows the time per iteration, and the fourth column shows the time per matrix cone projection.
  • Figure 2: Results for sparse inverse covariance selection.
  • Figure 3: Results for robust PCA for $m = n$ (first row), $m = 2n$ (second row), and $m = 5n$ (third row).
  • Figure 4: Results for graph partitioning.
  • Figure 5: The average time to compute the eigenvalue or singular value decomposition (orange line) and the average time to project onto the spectral vector cone (blue line) for (a) experimental design, (b) sparse inverse covariance selection, (c) robust PCA with $m = n$, (d) robust PCA with $m = 2n$, (e) robust PCA with $m = 5n$, and (f) graph partitioning.

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2