Proximal methods for structured nonsmooth optimization over Riemannian submanifolds
Qia Li, Na Zhang, Junyu Feng, Hanwei Yan
TL;DR
This work develops a rigorous framework for structured nonsmooth (DC and fractional) optimization over Riemannian submanifolds, introducing MPGSA to obtain subsequential convergence to a problem's critical point and, under KL, full-sequence convergence. It further extends the approach with EMPGSA for cases where $h_2$ is the maximum of convex differentiable terms, guaranteeing subsequential convergence to lifted B-stationary points. Central to the analysis are manifold subdifferentials, KL calculus, and an auxiliary function that ties together the augmented Lagrangian-like components with the manifold geometry. The proposed methods enable efficient handling of nonsmooth regularizers and fractional objectives on manifolds, with preliminary numerical demonstrations on sparse Fisher discriminant analysis showing improved speed and sparsity patterns. Collectively, these contributions advance practical and theoretical capabilities for nonsmooth optimization on embedded submanifolds, broadening applicability in machine learning and scientific computing.
Abstract
In this paper, we consider a class of structured nonsmooth optimization problems over an embedded submanifold of a Euclidean space, where the first part of the objective is the sum of a difference-of-convex (DC) function and a smooth function, while the remaining part is a weakly convex function over a smooth function. This model problem has many important applications in machine learning and scientific computing, for example, the sparse Fisher discriminant analysis. We propose a manifold proximal-gradient-subgradient algorithm (MPGSA) and show that under mild conditions any accumulation point of the solution sequence generated by it is a critical point of the underlying problem. By assuming the Kurdyka-Łojasiewicz property of an auxiliary function, we further establish the convergence of the full sequence generated by MPGSA under some suitable conditions. When the second component of the DC function involved is the maximum of finite continuously differentiable convex functions, we also propose an enhanced MPGSA with guaranteed subsequential convergence to a lifted B-stationary points of the optimization problem. Finally, some preliminary numerical experiments are conducted to illustrate the efficiency of the proposed algorithms.