Riemannian Zeroth-Order Gradient Estimation with Structure-Preserving Metrics for Geodesically Incomplete Manifolds

TL;DR

This work addresses zeroth-order optimization on Riemannian manifolds with geodesic incompleteness by introducing a structure-preserving metric g' that is geodesically complete and conformally related to the original metric g, while preserving ε-stationarity. It develops an intrinsic zeroth-order gradient estimator for non-Euclidean metrics and proves a curvature-dependent MSE bound, together with an unbiased g-unit sphere sampling method via rejection sampling. The authors establish convergence guarantees for Riemannian SGD under the non-Euclidean metric and show how, under suitable conditions, ε-stationarity with respect to g' implies ε-stationarity with respect to g, thus matching known complexity results in the geodesically complete setting. Empirical results on synthetic problems and a mesh-optimization task corroborate the theory, demonstrating stable convergence and practical viability even when geodesic completeness is absent.

Abstract

In this paper, we study Riemannian zeroth-order optimization in settings where the underlying Riemannian metric $g$ is geodesically incomplete, and the goal is to approximate stationary points with respect to this incomplete metric. To address this challenge, we construct structure-preserving metrics that are geodesically complete while ensuring that every stationary point under the new metric remains stationary under the original one. Building on this foundation, we revisit the classical symmetric two-point zeroth-order estimator and analyze its mean-squared error from a purely intrinsic perspective, depending only on the manifold's geometry rather than any ambient embedding. Leveraging this intrinsic analysis, we establish convergence guarantees for stochastic gradient descent with this intrinsic estimator. Under additional suitable conditions, an $ε$-stationary point under the constructed metric $g'$ also corresponds to an $ε$-stationary point under the original metric $g$, thereby matching the best-known complexity in the geodesically complete setting. Empirical studies on synthetic problems confirm our theoretical findings, and experiments on a practical mesh optimization task demonstrate that our framework maintains stable convergence even in the absence of geodesic completeness.

Figures (7)

• Figure 1: Geodesic contours centered at $p=(0.2,0.2,0.6)$ under the Euclidean metric (\ref{['fig:euclidean']}) and three structure-preserving metrics (\ref{['fig:conformal-metrics']}). Radii range from $0.1$ to $0.9$ in steps of $0.15$. Under each structure-preserving metric, geodesics from $p$ never exit the probability simplex, regardless of direction or length.
• Figure 2: Illustration of sampling on the unit sphere induced by the non-Euclidean Riemannian metric $g$. The naïve rescaling sampler (Left Panel) produces a visibly non-uniform distribution, leading to a biased estimator. Our rejection sampler (Right Panel) presented in \ref{['alg:sampling']} eliminates the bias and yields an even, truly uniform distribution.
• Figure 3: The impact of sampling bias on the convergence of Riemannian zeroth-order SGD.
• Figure 4: The impact of sectional curvatures on the gradient estimation accuracy.
• Figure 5: The leftmost panel illustrates an invalid optimization step on a mesh node; it crosses the edge, causing potential error in the external PDE solver. Figure (a) illustrates the Soft Projection approach, which resolves the issue by repeatedly reducing the perturbation stepsize $\mu$ along the perturbation direction $v$ until the movement becomes valid. Figure (b) shows the Reversion approach, which instead handles invalid steps by reverting to the original position. Figure (c) takes the advantage of the structure-preserving metric, which twists the underlying Riemannian structure ensuring that the perturbation won't move the point out of the domain.

Theorems & Definitions (48)

• Definition 2.5
• Theorem 2.6
• proof
• Theorem 2.7
• Proposition 2.8
• Theorem 2.9
• proof
• Corollary 2.10
• proof
• Definition B.1: $n$-Euclidean metric