Improved Approximation Algorithms for Low-Rank Problems Using Semidefinite Optimization

TL;DR

This work extends the Goemans-Williamson relax-then-round paradigm to semi-orthogonal and general low-rank optimization via new semidefinite relaxations and a novel sampling/rounding scheme. It delivers a purely multiplicative guarantee for orthogonality-constrained problems and shows how to tighten relaxations by eliminating many PSD variables, enabling scalable computation. The paper also generalizes the approach to matrix completion and reduced-rank regression, offering compact formulations, stronger relaxations, and extensive numerical validation. Overall, the framework provides scalable, principled approximations for challenging low-rank problems with practical impact on tasks like matrix completion and sparse regression.

Abstract

Inspired by the impact of the Goemans-Williamson algorithm on combinatorial optimization, we construct an analogous relax-then-round strategy for low-rank optimization problems. First, for orthogonally constrained quadratic optimization problems, we derive a semidefinite relaxation and a randomized rounding scheme that obtains provably near-optimal solutions, building on the blueprint from Goemans and Williamson for the Max-Cut problem. For a given $n \times m$ semi-orthogonal matrix, we derive a purely multiplicative approximation ratio for our algorithm, and show that it is never worse than $\max(2/(πm), 1/(π(\log (2m)+1)))$. We also show how to compute a tighter constant for a finite $(n,m)$ by solving a univariate optimization problem. We then extend our approach to generic low-rank optimization problems by developing new semidefinite relaxations that are both tighter and more broadly applicable than those in prior works. Although our original proposal introduces large semidefinite matrices as decision variables, we show that most of the blocks in these matrices can be safely omitted without altering the optimal value, hence improving the scalability of our approach. Using several examples (including matrix completion, basis pursuit, and reduced-rank regression), we show how to reduce the size of our relaxation even further. Finally, we numerically illustrate the effectiveness and scalability of our relaxation and sampling scheme on orthogonally constrained quadratic optimization and matrix completion problems.

Figures (9)

• Figure 1: Average performance ratio $\langle \bm{A}, \operatorname{vec}(\bm{Q})\operatorname{vec}(\bm{Q})^\top \rangle / \langle \bm{A}, \bm{W}^\star \rangle$ over $N=100$ generated solutions for different feasibility heuristics. Note that the method of burer2023strengthened is deterministic (always returns the same solution for a given instance). For each value of $m$, results are averaged over 5 instances.
• Figure 2: Performance ratio $\langle \bm{A}, \operatorname{vec}(\bm{Q})\operatorname{vec}(\bm{Q})^\top \rangle / \langle \bm{A}, \bm{W}^\star \rangle$ of the best out of $N=100$ solutions for different feasibility heuristics. Note that the method of burer2023strengthened only returns one solution. For each value of $m$, results are averaged over 5 instances.
• Figure 3: Relative gap obtained with different relaxations of the regularized matrix completion problem as we vary $\gamma$. We fix $n=8$. Results are averaged over 10 replications.
• Figure 4: Average quality of GW rounding as we vary $\gamma$ for rounding the full Shor relaxation ("GW-Full") and the reduced relaxation with and without projecting $\bm{W}_{y,y}$ ("GW-Red-Proj", "GW-Red-NoProj").
• Figure 5: Average quality of feasible methods as we vary $\gamma$, for GW rounding on the full Shor relaxation ("GW-Full"), on the reduced relaxation with projecting $\bm{W}_{y,y}$ ("GW-Red-Proj"), greedily rounding the matrix perspective relaxation ("MPRT-GD"), and alternating minimization ("AM").

Theorems & Definitions (29)

• Proposition 1
• Lemma 1
• Theorem 1
• Proposition 2
• Theorem 2
• Proposition 3
• Remark 1
• Proposition 4
• Remark 2
• Theorem 3