New Understandings and Computation on Augmented Lagrangian Methods for Low-Rank Semidefinite Programming
Lijun Ding, Haihao Lu, Jinwen Yang
TL;DR
The paper addresses the theoretical foundations of the ALM-BM approach for large-scale, low-rank SDPs by proving that, under primal simplicity of the original SDP and proximity of the dual variable to a dual optimum, ALM subproblems inherit strong duality, strict complementarity, and low-rank structure, along with a nondegenerate quadratic growth condition that is independent of the dual iterate. It further shows that the nonconvex Burer–Monteiro reformulation admits global solutions via simple gradient descent with linear convergence when started near the optimal point and with dual proximity, providing rigorous convergence guarantees in a nonconvex setting. Motivated by these results, the authors introduce ALORA, a rank-adaptive augmented Lagrangian method that uses spectral information and negative-curvature exploration to adjust rank dynamically, and implement a GPU-accelerated solver that scales to SDPs with tens of millions of variables, demonstrated on MaxCut and matrix completion. They also present illustrative examples underscoring the necessity of the local assumptions and discuss the practical impact of their approach, highlighting improved scalability and reliability for solving large-scale SDP relaxations in applications.
Abstract
Augmented Lagrangian Method (ALM) combined with Burer-Monteiro (BM) factorization, dubbed ALM-BM, offers a powerful approach for solving large-scale low-rank semidefinite programs (SDPs). Despite its empirical success, the theoretical understandings of the resulting non-convex ALM-BM subproblems, particularly concerning their structural properties and efficient subproblem solvability by first-order methods, still remain limited. This work addresses these notable gaps by providing a rigorous theoretical analysis. We demonstrate that, under appropriate regularity of the original SDP, termed as primal simplicity, ALM subproblems inherit crucial properties such as low-rankness and strict complementarity when the dual variable is localized. Furthermore, ALM subproblems are shown to enjoy a quadratic growth condition, building on which we prove that the non-convex ALM-BM subproblems can be solved to global optimality by gradient descent, achieving linear convergence under conditions of local initialization and dual variable proximity. Through illustrative examples, we further establish the necessity of these local assumptions, revealing them as inherent characteristics of the problem structure. Motivated by these theoretical insights, we propose ALORA, a rank-adaptive augmented Lagrangian method that builds upon the ALM-BM framework, which dynamically adjusts the rank using spectral information and explores negative curvature directions to navigate the nonconvex landscape. Exploiting modern GPU computing architectures, ALORA exhibits strong numerical performance, solving SDPs with tens of millions of dimensions in hundreds of seconds.