Stochastic optimization over proximally smooth sets

TL;DR

The paper develops a comprehensive framework for stochastic optimization of weakly convex functions over proximally smooth constraint sets, extending model-based stochastic methods beyond convex feasibility. By combining stochastic models, locally defined set approximations, and retractions back to the original set, it obtains finite-time guarantees via the Moreau envelope and a proximal-stationarity measure. The authors provide two concrete instantiations—tangent-space (Riemannian) approximations for manifolds and inner approximations for functional constraints—along with rigorous proofs of convergence under suitable regularity and error-bound conditions. This work bridges nonsmooth stochastic optimization with nonconvex constraint geometry, enabling tractable, provably efficient algorithms for optimization over manifolds and nonconvex constraint sets with weakly convex objectives.

Abstract

We introduce a class of stochastic algorithms for minimizing weakly convex functions over proximally smooth sets. As their main building blocks, the algorithms use simplified models of the objective function and the constraint set, along with a retraction operation to restore feasibility. All the proposed methods come equipped with a finite time efficiency guarantee in terms of a natural stationarity measure. We discuss consequences for nonsmooth optimization over smooth manifolds and over sets cut out by weakly-convex inequalities.

Figures (3)

• Figure 1: One-sided models for the function $f(x,\xi)=|x^2-1|$ at the point $x=0.5$, appearing in the same order as in Table \ref{['table:functionmodels']}.
• Figure 2: Unit Sphere and its tangent approximations.
• Figure 3: Both of the figures depict the feasible region $\mathcal{X}=\{(x,y): g_1(x,y)\leq 0, g_2(x,y)\leq 0\}$, where we define the two quadratics $g_1(x,y)=x^2-y$ and $g_2(x,y)=y - \tfrac{1}{5} x^2-\tfrac{4}{5}$. The basepoints where the set approximations are formed, depicted in red, are $(x_0,y_0)\in \{(1,1),(-0.7,0.8)\}$. The region with the solid boundary is $\{(x,y):g_i(x,y) + \tfrac{1}{4} \|(x,y)-(x_0,y_0)\|^2\leq 0 ~\forall i=1,2\}$. The region with the dashed boundary is $\{(x,y):g_i(x_0,y_0)+\langle \nabla g(x_0,y_0),(x,y)-(x_0,y_0)\rangle + 1.1\|(x,y)-(x_0,y_0)\|^2\leq 0~\forall i=1,2\}$.

Theorems & Definitions (28)

• Lemma 3.1: Proximally smooth sets
• Lemma 3.2
• Lemma 3.3: Three-point inequality
• Proof 1
• Theorem 4.1: Convergence guarantees
• Lemma 4.2: One-step improvement
• Proof 2
• Proof 3: Proof of Theorem \ref{['thm:simple_result']}
• Definition 5.1: Set-approximation
• Lemma 5.2