Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MSINO: Curvature-Aware Sobolev Optimization for Manifold Neural Networks

Suresan Pareth

TL;DR

The method replaces standard Euclidean derivative supervision with a covariant Sobolev loss that aligns gradients using parallel transport and improves stability via a Laplace Beltrami smoothness regularization term.

Abstract

We introduce Manifold Sobolev Informed Neural Optimization (MSINO), a curvature aware training framework for neural networks defined on Riemannian manifolds. The method replaces standard Euclidean derivative supervision with a covariant Sobolev loss that aligns gradients using parallel transport and improves stability via a Laplace Beltrami smoothness regularization term. Building on classical results in Riemannian optimization and Sobolev theory on manifolds, we derive geometry dependent constants that yield (i) a Descent Lemma with a manifold Sobolev smoothness constant, (ii) a Sobolev Polyak Lojasiewicz inequality giving linear convergence guarantees for Riemannian gradient descent and stochastic gradient descent under explicit step size bounds, and (iii) a two step Newton Sobolev method with local quadratic contraction in curvature controlled neighborhoods. Unlike prior Sobolev training in Euclidean space, MSINO provides training time guarantees that explicitly track curvature and transported Jacobians. Applications include surface imaging, physics informed learning settings, and robotics on Lie groups such as SO(3) and SE(3). The framework unifies value and gradient based learning with curvature aware convergence guarantees for neural training on manifolds.

MSINO: Curvature-Aware Sobolev Optimization for Manifold Neural Networks

TL;DR

The method replaces standard Euclidean derivative supervision with a covariant Sobolev loss that aligns gradients using parallel transport and improves stability via a Laplace Beltrami smoothness regularization term.

Abstract

We introduce Manifold Sobolev Informed Neural Optimization (MSINO), a curvature aware training framework for neural networks defined on Riemannian manifolds. The method replaces standard Euclidean derivative supervision with a covariant Sobolev loss that aligns gradients using parallel transport and improves stability via a Laplace Beltrami smoothness regularization term. Building on classical results in Riemannian optimization and Sobolev theory on manifolds, we derive geometry dependent constants that yield (i) a Descent Lemma with a manifold Sobolev smoothness constant, (ii) a Sobolev Polyak Lojasiewicz inequality giving linear convergence guarantees for Riemannian gradient descent and stochastic gradient descent under explicit step size bounds, and (iii) a two step Newton Sobolev method with local quadratic contraction in curvature controlled neighborhoods. Unlike prior Sobolev training in Euclidean space, MSINO provides training time guarantees that explicitly track curvature and transported Jacobians. Applications include surface imaging, physics informed learning settings, and robotics on Lie groups such as SO(3) and SE(3). The framework unifies value and gradient based learning with curvature aware convergence guarantees for neural training on manifolds.
Paper Structure (28 sections, 12 theorems, 145 equations, 7 figures, 4 tables, 1 algorithm)

This paper contains 28 sections, 12 theorems, 145 equations, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Preliminaries and Notation
  2. Neural parameterization.
  3. Parallel transport along geodesics.
  4. Neural parameterization and transported differentials.
  5. Manifold Sobolev--Informed Loss
  6. Discrete approximation.
  7. Computational complexity.
  8. Geometry- and Model-Dependent Constants
  9. Descent Lemma and PL Geometry
  10. Convergence of Riemannian GD/SGD
  11. Two-Step Newton--Sobolev Method on Manifolds
  12. Practical estimation.
  13. Variance-Aware Sobolev Weighting
  14. Algorithm: Riemannian Sobolev-SGD
  15. Curvature-Explicit Sobolev--PL Constant
  16. ...and 13 more sections

Key Result

Lemma 3.1

Under Assumptions assump:geometry and assump:spectral, for any $\theta,\theta'$ such that all evaluations $u_\theta(x), u_{\theta'}(x)$ with $x \in \operatorname{supp}(D)$ lie in a common geodesically convex set $U \subset M$, we have where $L^M_{\mathrm{sob}}(\theta)=C(M,g)(1+\lambda)\,\mathcal{S}(\theta)$ is the constant defined in Definition def:Lsob.

Figures (7)

  • Figure 1: Unified MSINO metrics across all tasks. Columns show total Sobolev loss, adaptive scheduling $\lambda_k$, contraction ratio $\rho_k$, and the smoothness constant $L^M_{\mathrm{sob}}(\theta_k)$.
  • Figure 2: MSINO convergence on cortical mesh.
  • Figure 3: Predicted cortical thickness on the mesh.
  • Figure 4: Prediction vs ground truth vs error.
  • Figure 5: Climate experiment: convergence on $\mathbb{S}^2$. (Left) Total Sobolev loss $L_{\mathrm{Sob},M}(\theta_k)$. (Right) Contraction ratio $\rho_k$, illustrating stable behaviour compatible with Theorems \ref{['thm:gd_convergence']} and \ref{['thm:newton_convergence']}.
  • ...and 2 more figures

Theorems & Definitions (34)

  • Definition 1.1: Manifold Sobolev space Hebey96
  • Definition 2.1: Manifold Sobolev loss
  • Remark 2.2
  • Definition 2.3: Sobolev smoothness constant
  • Lemma 3.1: Riemannian Sobolev Descent Lemma
  • proof
  • Definition 3.2: Manifold Sobolev--PL inequality
  • Remark 3.3
  • proof : Proof sketch
  • Theorem 4.1: Linear convergence of Riemannian gradient descent under Sobolev--PL
  • ...and 24 more