Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Geometry-Preserving Neural Architectures on Manifolds with Boundary

Karthik Elamvazhuthi, Shiba Biswal, Kian Rosenblum, Arushi Katyal, Tianli Qu, Grady Ma, Rishi Sonthalia

TL;DR

The paper tackles the challenge of training neural models that respect hard geometric constraints by introducing geometry-preserving architectures for manifolds with boundary. It unifies two design paradigms—Intermediate Augmented Architectures (IAA), which interleave constraint-preserving updates with ambient layers, and Final Augmented Architectures (FAA), which apply geometry guarantees only at the output—and develops both projected and exponential-update variants, including a Lie-group specialization. It provides universal approximation guarantees for constrained neural ODEs and for FAA variants, along with a diffusion-based method to learn projections when analytic maps are unavailable. Empirically, the authors demonstrate exact feasibility for analytic geometric updates and strong performance for learned projections across dynamics on $S^2$, $SO(3)$, the unit disk, and SE(3)-structured protein data, highlighting the practical impact of enforcing geometry during learning. These geometry-aware designs offer principled, invariant mappings with improved tradeoffs between accuracy and constraint satisfaction, suitable for physics- and structure-constrained applications.

Abstract

Preserving geometric structure is important in learning. We propose a unified class of geometry-aware architectures that interleave geometric updates between layers, where both projection layers and intrinsic exponential map updates arise as discretizations of projected dynamical systems on manifolds (with or without boundary). Within this framework, we establish universal approximation results for constrained neural ODEs. We also analyze architectures that enforce geometry only at the output, proving a separate universal approximation property that enables direct comparison to interleaved designs. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Experiments on dynamics over S^2 and SO(3), and diffusion on S^{d-1}-valued features demonstrate exact feasibility for analytic updates and strong performance for learned projections

Geometry-Preserving Neural Architectures on Manifolds with Boundary

TL;DR

The paper tackles the challenge of training neural models that respect hard geometric constraints by introducing geometry-preserving architectures for manifolds with boundary. It unifies two design paradigms—Intermediate Augmented Architectures (IAA), which interleave constraint-preserving updates with ambient layers, and Final Augmented Architectures (FAA), which apply geometry guarantees only at the output—and develops both projected and exponential-update variants, including a Lie-group specialization. It provides universal approximation guarantees for constrained neural ODEs and for FAA variants, along with a diffusion-based method to learn projections when analytic maps are unavailable. Empirically, the authors demonstrate exact feasibility for analytic geometric updates and strong performance for learned projections across dynamics on , , the unit disk, and SE(3)-structured protein data, highlighting the practical impact of enforcing geometry during learning. These geometry-aware designs offer principled, invariant mappings with improved tradeoffs between accuracy and constraint satisfaction, suitable for physics- and structure-constrained applications.

Abstract

Preserving geometric structure is important in learning. We propose a unified class of geometry-aware architectures that interleave geometric updates between layers, where both projection layers and intrinsic exponential map updates arise as discretizations of projected dynamical systems on manifolds (with or without boundary). Within this framework, we establish universal approximation results for constrained neural ODEs. We also analyze architectures that enforce geometry only at the output, proving a separate universal approximation property that enables direct comparison to interleaved designs. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Experiments on dynamics over S^2 and SO(3), and diffusion on S^{d-1}-valued features demonstrate exact feasibility for analytic updates and strong performance for learned projections
Paper Structure (76 sections, 8 theorems, 180 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 76 sections, 8 theorems, 180 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation.
  3. Geometry Preserving Architectures
  4. Intermediate Augmented Architectures
  5. Neural ODEs.
  6. Projected IAAs.
  7. Exponential IAAs.
  8. Exponential IAAs on Lie Groups.
  9. Final Augmented Architectures
  10. Projected FAAs.
  11. Exponential FAAs
  12. Learned Projections via Flow Matching
  13. Numerical Experiments
  14. Experiment 2: Learning dynamics on manifolds.
  15. Notation
  16. ...and 61 more sections

Key Result

Theorem 2.2

Suppose $M$ is uniformly prox-regular. Let $F,f_\theta:[0,T]\times M\to\mathbb{R}^d$ be such that it is continuous, satisfy assump and Then, with $\mathscr{L}_\alpha:=\mathscr{L}+\frac{4U^2}{\alpha}+1$, where $\alpha$ is the reach of $M$, one has

Figures (2)

  • Figure 1: Overview of the geometry-preserving architectures, illustrating interleaved geometric augmentations, intrinsic updates, and final-layer projections.
  • Figure 2: Analysis of projection performance: (a) shows the Mean Squared Error, and (b) shows the distance to the manifold relative to the noise standard deviation $\sigma$. More details on the distance used can be found in Appendix \ref{['app:dist']}

Theorems & Definitions (12)

  • Theorem 2.2
  • Theorem 2.3
  • Theorem 2.4
  • Theorem 2.5
  • Theorem 2.1
  • proof
  • Theorem 2.1
  • proof
  • Theorem 2.1
  • proof
  • ...and 2 more