Projected subgradient methods for paraconvex optimization: Application to robust low-rank matrix recovery

TL;DR

The paper extends projected subgradient methods to the broad class of $ u$-paraconvex nonsmooth, nonconvex objectives under a Hölderian error bound, providing a unified convergence-rate framework for several step-size schemes. It establishes fundamental properties of $ u$-paraconvex functions, including local Lipschitzness and saddle-point structure, and shows that linear convergence is achievable under suitable step-sizes. The authors develop and analyze projected subgradient methods with constant, nonsummable, square-summable, geometrically decaying, and Scaled Polyak step-sizes, proving subsequential/global convergence and explicit rates. They validate the approach on robust low-rank matrix recovery problems—robust matrix completion, image inpainting, and RNMF-based tasks—where Scaled Polyak often outperforms traditional strategies, highlighting practical impact for large-scale, nonsmooth nonconvex optimization.

Abstract

This paper is devoted to the class of paraconvex functions and presents some of its fundamental properties, characterization, and examples that can be used for their recognition and optimization. Next, the convergence analysis of the projected subgradient methods with several step-sizes (i.e., constant, nonsummable, square-summable but not summable, geometrically decaying, and Scaled Polyak's step-sizes) to global minima for this class of functions is studied. In particular, the convergence rate of the proposed methods is investigated under paraconvexity and the Hölderian error bound condition, where the latter is an extension of the classical error bound condition. The preliminary numerical experiments on several robust low-rank matrix recovery problems (i.e., robust matrix completion, image inpainting, robust nonnegative matrix factorization, robust matrix compression) indicate promising behavior for these projected subgradient methods, validating our theoretical foundations.

Figures (10)

• Figure 1: Comparisons of the function $h(x)$ and its modified versions demonstrating $\nu$-paraconvexity. The left plot shows $h(x)$ (in black), $h(x)+|x|^{1.5}$ (in green), highlighting the convexity of $h(x)+|x|^{1.5}$ which implies $h$ is $0.5$-paraconvex. The right plot shows $h(x)$ alongside $h(x)+|x|^{2}$ (in purple), illustrating that $h$ is not $1$-paraconvex.
• Figure 2: Stationary points of the function $h(x,y)= x^2 + (y^2 - 1)^2$: global minima at $(0,\pm 1)$ (red points) and a saddle point at $(0,0)$ (white point).
• Figure 3: Convergence of the loss function (log scale) over 1000 iterations for the MovieLens 100K dataset, comparing four step-size strategies: Polyak, Scaled Polyak, Diminishing, and Decaying using two initial points.
• Figure 4: The first row displays the original images: "Man" (a), "Kiel" (b), "Lighthouse" (c), and "Houses" (d). The second row shows the corresponding corrupted images (e–h) with $40\%$ random uniform noise applied.
• Figure 5: Qualitative comparison of image inpainting results for four corrupted images (lighthouse, man, kiel, and houses) using Polyak, Scaled Polyak, Diminishing, and Decaying step-size strategies. Each column shows the reconstructions for the step-size method.

Theorems & Definitions (21)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof