Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Scalable Semidefinite Programming: Optimal Metric ADMM with A Worst-case Performance Guarantee

Yifan Ran, Stefan Vlaski, Wei Dai

TL;DR

This work aims to achieve extra acceleration for ADMM by appealing to a non-Euclidean metric space, while maintaining everything in closed-form expressions, and observes that the scalability property is significantly improved.

Abstract

Despite the numerous uses of semidefinite programming (SDP) and its universal solvability via interior point methods (IPMs), it is rarely applied to practical large-scale problems. This mainly owes to the computational cost of IPMs that increases in a bad exponential way with the data size. While first-order algorithms such as ADMM can alleviate this issue, but the scalability improvement appears far not enough. In this work, we aim to achieve extra acceleration for ADMM by appealing to a non-Euclidean metric space, while maintaining everything in closed-form expressions. The efficiency gain comes from the extra degrees of freedom of a variable metric compared to a scalar step-size, which allows us to capture some additional ill-conditioning structures. On the application side, we consider the quadratically constrained quadratic program (QCQP), which naturally appears in an SDP form after a dualization procedure. This technique, known as semidefinite relaxation, has important uses across different fields, particularly in wireless communications. Numerically, we observe that the scalability property is significantly improved. Depending on the data generation process, the extra acceleration can easily surpass the scalar-parameter efficiency limit, and the advantage is rapidly increasing as the data conditioning becomes worse.

Towards Scalable Semidefinite Programming: Optimal Metric ADMM with A Worst-case Performance Guarantee

TL;DR

This work aims to achieve extra acceleration for ADMM by appealing to a non-Euclidean metric space, while maintaining everything in closed-form expressions, and observes that the scalability property is significantly improved.

Abstract

Despite the numerous uses of semidefinite programming (SDP) and its universal solvability via interior point methods (IPMs), it is rarely applied to practical large-scale problems. This mainly owes to the computational cost of IPMs that increases in a bad exponential way with the data size. While first-order algorithms such as ADMM can alleviate this issue, but the scalability improvement appears far not enough. In this work, we aim to achieve extra acceleration for ADMM by appealing to a non-Euclidean metric space, while maintaining everything in closed-form expressions. The efficiency gain comes from the extra degrees of freedom of a variable metric compared to a scalar step-size, which allows us to capture some additional ill-conditioning structures. On the application side, we consider the quadratically constrained quadratic program (QCQP), which naturally appears in an SDP form after a dualization procedure. This technique, known as semidefinite relaxation, has important uses across different fields, particularly in wireless communications. Numerically, we observe that the scalability property is significantly improved. Depending on the data generation process, the extra acceleration can easily surpass the scalar-parameter efficiency limit, and the advantage is rapidly increasing as the data conditioning becomes worse.
Paper Structure (35 sections, 12 theorems, 95 equations, 2 figures, 3 algorithms)

This paper contains 35 sections, 12 theorems, 95 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notations
  3. ADMM algorithm
  4. Basic iterates
  5. Metric space iterates
  6. Semidefinite programming
  7. Preliminary: a unified treatment
  8. Primal: differentiation & injectivity
  9. Towards extra efficiency
  10. Closed-form guaranteed metrics
  11. Definiteness invariant condition
  12. Construction: feasible choices
  13. Closed-form metric solver
  14. Optimal metric selection
  15. Structure I: optimal parameters
  16. ...and 20 more sections

Key Result

Proposition 3.1

Consider a metric space environment $\mathscr H_\mathcal{M}$, with $\langle \cdot, \, \mathcal{M}\,\cdot \rangle = \langle \mathcal{S}\,\cdot, \, \mathcal{S}\,\cdot \rangle$. Suppose the decomposed metric $\mathcal{S}$ admits the following definiteness-invariant characterization: with $\bm{Z} \in \mathbb{S}^{N}$ being an arbitrary symmetric matrix, where $\mathbb{S}^{N}_+ = \{ \bm{X} \in \mat

Figures (2)

  • Figure 1: Iteration number complexity against data dimension.
  • Figure 2: Iteration number complexity performance.

Theorems & Definitions (20)

  • Proposition 3.1
  • proof
  • Proposition 3.2
  • Proposition 3.3
  • proof
  • Lemma 4.1
  • Corollary 4.1: SDP choice
  • Proposition 4.1
  • proof
  • Proposition 4.2
  • ...and 10 more