A New Inexact Manifold Proximal Linear Algorithm with Adaptive Stopping Criteria

TL;DR

The paper addresses nonsmooth, nonconvex composite optimization over embedded manifolds by introducing IManPL, an inexact manifold proximal linear method with adaptive stopping criteria. By solving subproblems inexactly via APG (or ASSN for identity mappings) and employing low/high accuracy controls (LACC/HACC), the method achieves $O(1/\epsilon^2)$ main iterations and $O(1/\epsilon^3)$ first-order oracle complexity, with accumulation points guaranteed to be stationary. Theoretical results are complemented by numerical experiments on sparse spectral clustering and sparse PCA, where IManPL outperforms existing manifold-based methods in CPU time while maintaining accuracy. The work advances adaptive inexactness in manifold optimization, enabling scalable solutions to large-scale problems in statistics and signal processing.

Abstract

This paper proposes a new inexact manifold proximal linear (IManPL) algorithm for solving nonsmooth, nonconvex composite optimization problems over an embedded submanifold. At each iteration, IManPL solves a convex subproblem inexactly, guided by two adaptive stopping criteria. We establish convergence guarantees and show that IManPL achieves the best first-order oracle complexity for solving this class of problems. Numerical experiments on sparse spectral clustering and sparse principal component analysis demonstrate that our methods outperform existing approaches.

Theorems & Definitions (18)

• Definition 1
• Lemma 1: Weak Convexity, Lemma 3.2 in drusvyatskiy2019efficiency
• Lemma 2
• proof
• Lemma 3
• proof
• Lemma 4
• proof
• Lemma 5: Sufficient Decrease
• proof