Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intrinsic Lorentz Neural Network

Xianglong Shi, Ziheng Chen, Yunhan Jiang, Nicu Sebe

TL;DR

The ILNN is a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model, replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature.

Abstract

Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. To address it, we propose the \emph{Intrinsic Lorentz Neural Network} (ILNN), a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces a novel \emph{point-to-hyperplane} fully connected layer (FC), replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature. Around this fundamental layer, we design intrinsic modules: GyroLBN, a Lorentz batch normalization that couples gyro-centering with gyro-scaling, consistently outperforming both LBN and GyroBN while reducing training time. We additionally proposed a gyro-additive bias for the FC output, a Lorentz patch-concatenation operator that aligns the expected log-radius across feature blocks via a digamma-based scale, and a Lorentz dropout layer. Extensive experiments conducted on CIFAR-10/100 and two genomic benchmarks (TEB and GUE) illustrate that ILNN achieves state-of-the-art performance and computational cost among hyperbolic models and consistently surpasses strong Euclidean baselines. The code is available at \href{https://github.com/Longchentong/ILNN}{\textcolor{magenta}{this url}}.

Intrinsic Lorentz Neural Network

TL;DR

The ILNN is a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model, replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature.

Abstract

Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. To address it, we propose the \emph{Intrinsic Lorentz Neural Network} (ILNN), a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces a novel \emph{point-to-hyperplane} fully connected layer (FC), replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature. Around this fundamental layer, we design intrinsic modules: GyroLBN, a Lorentz batch normalization that couples gyro-centering with gyro-scaling, consistently outperforming both LBN and GyroBN while reducing training time. We additionally proposed a gyro-additive bias for the FC output, a Lorentz patch-concatenation operator that aligns the expected log-radius across feature blocks via a digamma-based scale, and a Lorentz dropout layer. Extensive experiments conducted on CIFAR-10/100 and two genomic benchmarks (TEB and GUE) illustrate that ILNN achieves state-of-the-art performance and computational cost among hyperbolic models and consistently surpasses strong Euclidean baselines. The code is available at \href{https://github.com/Longchentong/ILNN}{\textcolor{magenta}{this url}}.
Paper Structure (67 sections, 2 theorems, 69 equations, 3 figures, 7 tables)

This paper contains 67 sections, 2 theorems, 69 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. related work
  3. Hyperbolic embeddings.
  4. Hyperbolic neural networks.
  5. Normalization in HNNs.
  6. Background
  7. Lorentz model.
  8. Gyrovector space.
  9. Intrinsic Lorentz neural network
  10. Point-to-hyperplane Lorentz fully‑connected layer
  11. Lorentz MLR.
  12. Lorentz fully‑connected layer (PLFC).
  13. Gyro‑bias.
  14. Discussion.
  15. Gyrogroup Lorentz Batch Normalization (GyroLBN)
  16. ...and 52 more sections

Key Result

Theorem 1

Let $\bm{x}\!\in\!\mathbb{L}^{n}_K$, $Z=\{\bm{z}_k\}_{k=1}^{m}\!\subset\!\mathbb{R}^{n}$ and $a=\{a_k\}_{k=1}^{m}\!\subset\!\mathbb{R}$. The point–to–hyperplane Lorentz fully connected layer${\operatorname{PLFC}_{K}\!:\mathbb{L}^{n}_K\to\mathbb{L}^{m}_K}$ is where $\bm{y}_s=(y_{s,1},\dots,y_{s,m})^{\!\top}$. In the flat‑space limit $K \!\to\! 0$, equation eq:plfc_def reduces to the Euclidean affi

Figures (3)

  • Figure 1: 2-dimensional Lorentz model in the 3-dimensional Minkowski space.
  • Figure 2: Embedding visualization of CIFAR-10 dataset in Poincaré and Tangent Space. Colors represent labels. HCNN (94.98, left) and ILNN (95.48, right).
  • Figure 3: Embedding visualization of CIFAR-100 dataset in Poincaré and Tangent Space. Colors represent labels. HCNN (77.67, left) and ILNN (78.64, right).

Theorems & Definitions (7)

  • Theorem 1: PLFC layer
  • Definition 1: Gyrogroup
  • Definition 2: Gyrovector space
  • proof
  • Definition 3
  • Theorem 2: Margin preservation and contraction of PLFC and LFC
  • proof