Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

General Constrained Matrix Optimization

Casey Garner, Gilad Lerman, Shuzhong Zhang

Abstract

This paper presents and analyzes the first matrix optimization model which allows general coordinate and spectral constraints. The breadth of problems our model covers is exemplified by a lengthy list of examples from the literature, including semidefinite programming, matrix completion, and quadratically constrained quadratic programs (QCQPs), and we demonstrate our model enables completely novel formulations of numerous problems. Our solution methodology leverages matrix factorization and constrained manifold optimization to develop an equivalent reformulation of our general matrix optimization model for which we design a feasible, first-order algorithm. We prove our algorithm converges to $(ε,ε)$-approximate first-order KKT points of our reformulation in $\mathcal{O}(1/ε^2)$ iterations. The method we developed applies to a special class of constrained manifold optimization problems and is one of the first which generates a sequence of feasible points which converges to a KKT point. We validate our model and method through numerical experimentation. Our first experiment presents a generalized version of semidefinite programming which allows novel eigenvalue constraints, and our second numerical experiment compares our method to the classical semidefinite relaxation approach for solving QCQPs. For the QCQP numerical experiments, we demonstrate our method is able to dominate the classical state-of-the-art approach, solving more than ten times as many problems compared to the standard solution procedure.

General Constrained Matrix Optimization

Abstract

This paper presents and analyzes the first matrix optimization model which allows general coordinate and spectral constraints. The breadth of problems our model covers is exemplified by a lengthy list of examples from the literature, including semidefinite programming, matrix completion, and quadratically constrained quadratic programs (QCQPs), and we demonstrate our model enables completely novel formulations of numerous problems. Our solution methodology leverages matrix factorization and constrained manifold optimization to develop an equivalent reformulation of our general matrix optimization model for which we design a feasible, first-order algorithm. We prove our algorithm converges to -approximate first-order KKT points of our reformulation in iterations. The method we developed applies to a special class of constrained manifold optimization problems and is one of the first which generates a sequence of feasible points which converges to a KKT point. We validate our model and method through numerical experimentation. Our first experiment presents a generalized version of semidefinite programming which allows novel eigenvalue constraints, and our second numerical experiment compares our method to the classical semidefinite relaxation approach for solving QCQPs. For the QCQP numerical experiments, we demonstrate our method is able to dominate the classical state-of-the-art approach, solving more than ten times as many problems compared to the standard solution procedure.

Paper Structure

This paper contains 29 sections, 17 theorems, 162 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivating Examples
  3. Generalized Semidefinite Programs
  4. Matrix Preconditioning
  5. Inverse Eigenvalue and Singular Value Problems and Constrained PCA
  6. Quadratically Constrained Quadratic Programs
  7. Structured Linear Regression
  8. Matrix Completion
  9. And More...
  10. An Equivalent Reformulation
  11. A General Block Manifold Model
  12. Riemannian Manifold Optimization Revisited
  13. Additional Definitions and Notation
  14. A Constrained Manifold Reformulation
  15. Approximate Stationary and KKT Points
  16. ...and 14 more sections

Key Result

Lemma 1

Given $\bm{X} \in \mathcal{S}^{n \times n}$, $\delta>0$, and a spectral decomposition of $\bm{X}$ such that $\bm{X} = \bm{Q}_x \operatorname{Diag}(\bm{\lambda}(\bm{X})) \bm{Q}_x^\top$ with $\bm{Q}_x \in \mathcal{O}(n,n)$, there exists $\delta'>0$ such that

Figures (2)

  • Figure 1: Displays the feasible region and level sets for an instance of \ref{['eq:QCQP_test_instances']} with $m=25$. The white region represents all of the feasible points outside the interior of the $m$ ellipsoids while the concentric circles centered at the origin are the level sets of the norm-squared objective function.
  • Figure 2: Displays the feasible points computed by each method for Test 8 with $m=50$. The yellow star is an estimated global optimal solution, the red circles are the randomized solutions generated from the SDR approach, the green circles are the candidate points generated from our method via (A2.), and magenta circles are the projected solutions obtained via (A1.). From left to right, the three panels display the results for \ref{['eq:QCQP_SCO']} with $\delta=10^{-1}$, $\delta=10^{-3}$, and $\delta=10^{-6}$.

Theorems & Definitions (54)

  • Remark 1.1
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Definition 1
  • Theorem 4
  • proof
  • ...and 44 more