Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Networks on Symmetric Spaces of Noncompact Type

Xuan Son Nguyen, Shuo Yang, Aymeric Histace

TL;DR

This work extends neural networks to symmetric spaces of noncompact type, unifying hyperplane concepts across hyperbolic spaces and SPD manifolds. It introduces a general point-to-hyperplane distance based on Busemann functions and group-theoretic structure, deriving closed-form expressions for higher rank spaces with G-invariant metrics. This distance enables new fully-connected layers and an attention mechanism that operate directly on the manifold, and the authors validate the approach on image classification, EEG classification, image generation, and natural language inference. The results demonstrate the potential of symmetric-space neural networks to capture hierarchical and geometric structure in diverse data while offering competitive performance and compact representations.

Abstract

Recent works have demonstrated promising performances of neural networks on hyperbolic spaces and symmetric positive definite (SPD) manifolds. These spaces belong to a family of Riemannian manifolds referred to as symmetric spaces of noncompact type. In this paper, we propose a novel approach for developing neural networks on such spaces. Our approach relies on a unified formulation of the distance from a point to a hyperplane on the considered spaces. We show that some existing formulations of the point-to-hyperplane distance can be recovered by our approach under specific settings. Furthermore, we derive a closed-form expression for the point-to-hyperplane distance in higher-rank symmetric spaces of noncompact type equipped with G-invariant Riemannian metrics. The derived distance then serves as a tool to design fully-connected (FC) layers and an attention mechanism for neural networks on the considered spaces. Our approach is validated on challenging benchmarks for image classification, electroencephalogram (EEG) signal classification, image generation, and natural language inference.

Neural Networks on Symmetric Spaces of Noncompact Type

TL;DR

This work extends neural networks to symmetric spaces of noncompact type, unifying hyperplane concepts across hyperbolic spaces and SPD manifolds. It introduces a general point-to-hyperplane distance based on Busemann functions and group-theoretic structure, deriving closed-form expressions for higher rank spaces with G-invariant metrics. This distance enables new fully-connected layers and an attention mechanism that operate directly on the manifold, and the authors validate the approach on image classification, EEG classification, image generation, and natural language inference. The results demonstrate the potential of symmetric-space neural networks to capture hierarchical and geometric structure in diverse data while offering competitive performance and compact representations.

Abstract

Recent works have demonstrated promising performances of neural networks on hyperbolic spaces and symmetric positive definite (SPD) manifolds. These spaces belong to a family of Riemannian manifolds referred to as symmetric spaces of noncompact type. In this paper, we propose a novel approach for developing neural networks on such spaces. Our approach relies on a unified formulation of the distance from a point to a hyperplane on the considered spaces. We show that some existing formulations of the point-to-hyperplane distance can be recovered by our approach under specific settings. Furthermore, we derive a closed-form expression for the point-to-hyperplane distance in higher-rank symmetric spaces of noncompact type equipped with G-invariant Riemannian metrics. The derived distance then serves as a tool to design fully-connected (FC) layers and an attention mechanism for neural networks on the considered spaces. Our approach is validated on challenging benchmarks for image classification, electroencephalogram (EEG) signal classification, image generation, and natural language inference.
Paper Structure (113 sections, 11 theorems, 133 equations, 5 figures, 12 tables)

This paper contains 113 sections, 11 theorems, 133 equations, 5 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Hyperbolic Spaces
  4. Matrix Manifolds
  5. General Riemannian Manifolds
  6. Mathematical Background
  7. Hyperbolic Spaces and SPD Manifolds
  8. Hyperbolic Spaces
  9. SPD Manifolds
  10. Symmetric Spaces of Noncompact Type
  11. Geometric boundary
  12. Angular metric
  13. Busemann functions
  14. Horocycles
  15. Composite distances
  16. ...and 98 more sections

Key Result

Corollary 4.3

Let $\ominus$ and $\oplus$ be the Möbius subtraction $\ominus_M$ and Möbius addition $\oplus_M$ in $\mathbb{B}_m$, respectively, and let $\| \cdot \|_{\mathbb{S}}$ be the Euclidean norm $\| \cdot \|$ (see Appendix sec:mobius_gyrovector_spaces). Let $p \in \mathbb{B}_m$, $\xi \in \partial \mathbb{B}_

Figures (5)

  • Figure 1: The distance between a point $x$ and a hyperplane $\mathcal{H}^E_{\xi,p}$.
  • Figure 2: Our proposed attention block (a) and the network architecture for EEG classification (b).
  • Figure 3: The network architecture for natural language inference.
  • Figure 4: Comparison of Poincaré hyperplanes and our hyperplanes in the Poincaré disk model.
  • Figure 5: The boundary $\partial \mathbb{D}$ of the Poincaré disk model $\mathbb{D}$ is illustrated by the green circle. The boundary point $\xi$ is normal to both the horocycle $\eta$ and the basic horocycle $\eta_0$. The distance between the origin $o$ and the horocycle $\xi$ is $-d$.

Theorems & Definitions (24)

  • Definition 4.1: Hyperplanes on a Symmetric Space
  • Definition 4.2
  • Corollary 4.3
  • Proposition 4.4
  • Definition 4.5: Binary Operation
  • Definition 4.6: Inverse Operation
  • Definition 4.7: The Inner Product on Symmetric Spaces
  • Proposition 4.8
  • Proposition 4.9
  • Corollary 4.10
  • ...and 14 more