Convergence analysis for a variant of manifold proximal point algorithm based on Kurdyka-Łojasiewicz property
Peiran Yu, Liaoyuan Zeng, Ting Kei Pong
TL;DR
The paper studies convergence of a variant of the manifold proximal point algorithm with iteratively reweighted regularization (${\sf ManPPA}_{\sf rw}$) for a nonconvex sparsity model on the unit sphere, where $f(x)=\Phi(|Y^T x|)=\sum_{i=1}^p \phi(|[Y^T x]_i|)$ and $\mathcal{M}=\{x:\|x\|=1\}$. It proves global convergence of the generated sequence under Kurdyka–Łojasiewicz (KL) properties of appropriate potential functions, and relates the KL exponents across these functions, yielding explicit rates. In the special case $\phi(s)=s$, corresponding to the linear sparsity regularizer, the method reduces to the original ManPPA with a constant stepsize, and the authors establish linear convergence when the optimal value is positive and finite convergence under weak sharp minima, improving prior local quadratic results. The analysis extends to general $\phi$ via a joint potential $\widetilde{F}$, showing that KL-based convergence still holds and deriving rate implications from KL exponents, thereby providing a rigorous framework for global convergence and rate characterization of nonconvex sparsity-regularized manifold optimization.
Abstract
We incorporate an iteratively reweighted strategy in the manifold proximal point algorithm (ManPPA) in [12] to solve an enhanced sparsity inducing model for identifying sparse yet nonzero vectors in a given subspace. We establish the global convergence of the whole sequence generated by our algorithm by assuming the Kurdyka-Lojasiewicz (KL) properties of suitable potential functions. We also study how the KL exponents of the different potential functions are related. More importantly, when our enhanced model and algorithm reduce, respectively, to the model and ManPPA with constant stepsize considered in [12], we show that the sequence generated converges linearly as long as the optimal value of the model is positive, and converges finitely when the limit of the sequence lies in a set of weak sharp minima. Our results improve [13, Theorem 2.4], which asserts local quadratic convergence in the presence of weak sharp minima when the constant stepsize is small.