A Riemannian Proximal Newton-CG Method
Wen Huang, Wutao Si
TL;DR
This work addresses global convergence for nonsmooth optimization on Riemannian manifolds by marrying a truncated conjugate gradient solver with a Riemannian proximal Newton step, producing a RPN-CG method. It proves global convergence and local superlinear convergence under standard assumptions, and demonstrates that a hybrid RPN-CGH variant is robust to switching parameters. The approach shows superior practical performance on sparse PCA, CM, and CD problems compared to state-of-the-art proximal-gradient-type methods. Overall, the method provides a scalable, second-order algorithm for nonsmooth manifold optimization with strong theoretical guarantees and empirical efficiency.
Abstract
Recently, a Riemannian proximal Newton method has been developed for optimizing problems in the form of $\min_{x\in\mathcal{M}} f(x) + μ\|x\|_1$, where $\mathcal{M}$ is a compact embedded submanifold and $f(x)$ is smooth. Although this method converges superlinearly locally, global convergence is not guaranteed. The existing remedy relies on a hybrid approach: running a Riemannian proximal gradient method until the iterate is sufficiently accurate and switching to the Riemannian proximal Newton method. This existing approach is sensitive to the switching parameter. This paper proposes a Riemannian proximal Newton-CG method that merges the truncated conjugate gradient method with the Riemannian proximal Newton method. The global convergence and local superlinear convergence are proven. Numerical experiments show that the proposed method outperforms other state-of-the-art methods.