A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
Chuan He, Heng Huang, Zhaosong Lu
TL;DR
This work develops a Newton-CG based barrier-augmented Lagrangian framework for general nonconvex conic optimization, aiming to compute approximate second-order stationary points. By integrating a preconditioned Newton-CG subsolver with barrier and augmented-Lagrangian techniques, the authors derive worst-case iteration and operation complexity bounds, including $\widetilde{\mathcal{O}}(\varepsilon^{-11/2})$ inner iterations (and $\widetilde{\mathcal{O}}(\varepsilon^{-7/2})$ under a constraint qualification) to obtain an $(\varepsilon,\sqrt{\varepsilon})$-SOSP. The method is backed by a rigorous analysis of optimality conditions under a verifiable CQ and a comprehensive complexity framework for the subproblem solver and AL outer loop; preliminary numerical results on low-rank recovery and NMF problems show improved solution quality over first-order methods. Overall, the paper advances the theory and practice of scalable second-order methods for nonconvex conic programs, offering practical algorithms with provable guarantees for challenging constrained settings.
Abstract
In this paper we consider finding an approximate second-order stationary point (SOSP) of general nonconvex conic optimization that minimizes a twice differentiable function subject to nonlinear equality constraints and also a convex conic constraint. In particular, we propose a Newton-conjugate gradient (Newton-CG) based barrier-augmented Lagrangian method for finding an approximate SOSP of this problem. Under some mild assumptions, we show that our method enjoys a total inner iteration complexity of $\widetilde{\cal O}(ε^{-11/2})$ and an operation complexity of $\widetilde{\cal O}(ε^{-11/2}\min\{n,ε^{-5/4}\})$ for finding an $(ε,\sqrtε)$-SOSP of general nonconvex conic optimization with high probability. Moreover, under a constraint qualification, these complexity bounds are improved to $\widetilde{\cal O}(ε^{-7/2})$ and $\widetilde{\cal O}(ε^{-7/2}\min\{n,ε^{-3/4}\})$, respectively. To the best of our knowledge, this is the first study on the complexity of finding an approximate SOSP of general nonconvex conic optimization. Preliminary numerical results are presented to demonstrate superiority of the proposed method over first-order methods in terms of solution quality.