A Variable Smoothing for Weakly Convex Composite Minimization with Manifold Constraint via Parametrization
Keita Kume, Isao Yamada
TL;DR
This work tackles manifold-constrained nonsmooth optimization of $F=x=h+g\circ\mathfrak{S}$ over a manifold $C$ by leveraging a parametrization $\varphi: \mathcal{Y}\to C$ to reformulate the problem on $\mathcal{Y}$. It introduces a variable smoothing approach built on the Moreau envelope ${}^{\mu}g$ and constructs a smoothed surrogate $F^{\langle \mu\rangle}\circ\varphi$, enabling gradient-descent updates that converge asymptotically to a stationary point; a gradient-consistency property ties the smoothed gradient to the limiting subdifferential of the original objective. The authors establish convergence results and a first-order oracle complexity of $\mathcal{O}(\epsilon^{-3})$, with additional conditions ensuring that stationary points in the parameter space map to stationary points on $C$. Numerical experiments on Sparse Spectral Clustering and Sparse PCA demonstrate the method’s efficacy, especially when using weakly convex regularizers like MCP and a Cayley-based parametrization of the Stiefel manifold, offering competitive performance against established Riemannian nonsmooth algorithms. Overall, the paper advances single-loop smoothing techniques for nonconvex, nonsmooth problems on manifolds and broadens their applicability to nonlinear smooth mappings and weakly convex penalties.
Abstract
In this paper, we address a manifold constrained nonsmooth optimization problem involving the composition of a weakly convex function and a smooth mapping under the availability of a parametrization of the manifold. To find a stationary point of the target problem, we propose a variable smoothing-type algorithm by combining the ideas of (i) translating the constrained problem into a Euclidean optimization problem with a parametrization of the constraint set; (ii) exploiting a sequence of smoothed surrogate functions, of the cost function, given with the Moreau envelope of a weakly convex function. The proposed algorithm produces a vector sequence by the gradient descent update of a smoothed surrogate function at each iteration. In a case where the proximity operator of the weakly convex function is available, the proposed algorithm does not require any iterative solver for subproblems therein. By leveraging tools in the variational analysis, we show the so-called {\em gradient consistency property}, which is a key ingredient for smoothing-type algorithms, of the smoothed surrogate function used in this paper. Based on the gradient consistency property, we also establish an asymptotic convergence analysis for the proposed algorithm regarding a stationary point. Numerical experiments demonstrate the efficacy of the proposed algorithm.