Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated first-order methods for a class of semidefinite programs

Alex L. Wang, Fatma Kilinc-Karzan

TL;DR

A new storage-optimal first-order method, CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy, and shows how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer.

Abstract

This paper introduces a new storage-optimal first-order method (FOM), CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the exact QMP-like SDPs, is characterized by low-rank solutions, a priori knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementarity. Crucially, we show how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer. From an algorithmic standpoint, we show how to construct the necessary certificate and how to solve the minimax problem efficiently. We accompany our theoretical results with preliminary numerical experiments suggesting that CertSDP significantly outperforms current state-of-the-art methods on large sparse exact QMP-like SDPs.

Accelerated first-order methods for a class of semidefinite programs

TL;DR

A new storage-optimal first-order method, CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy, and shows how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer.

Abstract

This paper introduces a new storage-optimal first-order method (FOM), CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the exact QMP-like SDPs, is characterized by low-rank solutions, a priori knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementarity. Crucially, we show how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer. From an algorithmic standpoint, we show how to construct the necessary certificate and how to solve the minimax problem efficiently. We accompany our theoretical results with preliminary numerical experiments suggesting that CertSDP significantly outperforms current state-of-the-art methods on large sparse exact QMP-like SDPs.
Paper Structure (42 sections, 23 theorems, 116 equations, 5 figures, 6 tables, 2 algorithms)

This paper contains 42 sections, 23 theorems, 116 equations, 5 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem setup and assumptions
  3. Overview and outline of the paper
  4. Related work
  5. Storage-optimal/efficient FOMs.
  6. FOMs for the GTRS.
  7. Accelerated FOMs for non-smooth problems via saddle-point problems.
  8. Accelerated FOMs with inexact first-order information.
  9. Notation
  10. Strongly convex reformulations of $k$-exact SDPs
  11. Definitions and problem setup
  12. Identifying ${\mathbb{S}}^n$ with quadratic matrix functions
  13. A strongly convex reformulation of \ref{['eq:primal_dual_SDP']}
  14. Comparison of \ref{['eq:strong_convex_reform']} with the Burer--Monteiro approach
  15. Algorithms for strongly convex QMMPs
  16. ...and 27 more sections

Key Result

lemma 1

Suppose $M^*,\,Y^*\in{\mathbb{S}}^n_+$ have rank $n-k$ and $k$ respectively and that $\left\langle M^*,Y^* \right\rangle=0$. Let $W$ be an $(n-k)$-dimensional subspace. Then, $M^*_{W}\succ 0$ if and only if $Y^*_{W^\perp}\succ 0$.

Figures (5)

  • Figure 1: CertSDP (\ref{['alg:exact_sdp_qmmp']}) produces a series of iterates $\gamma^{(i)}\to\gamma^*$. For each $\gamma^{(i)}$, CertSDP constructs a ball ${\cal U}^{(i)}$ around $\gamma^{(i)}$. Intuitively, we want to pick ${\cal U}^{(i)}$ to be the largest ball around $\gamma^{(i)}$ for which we can solve the associated QMMP efficiently, in hopes of enclosing $\gamma^*$. We will thus choose ${\cal U}^{(i)}$ to satisfy certain regularity estimates (see \ref{['eq:r']} and \ref{['lem:parameters_for_qmmp']}). At the minimum, we will ensure $A(\gamma)\succeq \hat{\mu}/2$ for all $\gamma\in{\cal U}^{(i)}$.
  • Figure 2: Memory usage of different algorithms as a function of the size $n-k$. In this chart, we plot $0.0$ MB at $1.0$ MB (see \ref{['rem:virtual_size_memory']} for a discussion on measuring memory usage).
  • Figure 3: Comparison of convergence behavior between CertSDP (\ref{['alg:exact_sdp_qmmp']}), CSSDP, and SketchyCGAL. The first, second, and third rows show experiments with $n-k=10^3$, $10^4$, and $10^5$ respectively. The right subplots give zoomed-in views of the primal squared distance in CertSDP on the final call to \ref{['alg:cautious-agd']}.
  • Figure 4: Memory usage of different algorithms on our phase retrieval instances as a function of the size $n$. In this chart, we plot $0.0$ MB at $1.0$ MB (see \ref{['rem:virtual_size_memory']} for a discussion on measuring memory usage).
  • Figure 5: Comparison of convergence behavior between CertSDP (\ref{['alg:exact_sdp_qmmp']}), CSSDP, and SketchyCGAL on our phase retrieval instances. The first, second, and third rows show experiments with $n=30$, $100$, and $300$ respectively.

Theorems & Definitions (54)

  • definition 1
  • definition 2
  • remark 1
  • lemma 1
  • proof
  • remark 2
  • theorem 1
  • proof
  • remark 3
  • remark 4
  • ...and 44 more