Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equivariant Neural Networks for General Linear Symmetries on Lie Algebras

Chankyo Kim, Sicheng Zhao, Minghan Zhu, Tzu-Yuan Lin, Maani Ghaffari

TL;DR

This work tackles learning under broad linear symmetries by introducing Reductive Lie Neurons (ReLNs), which enforce exact adjoint-equivariance for the general linear group $GL(n)$ and its reductive subalgebras. The core idea is a learnable Ad-invariant bilinear form $ ilde B$ on the reductive algebra $gl(n)$ that remains nondegenerate, enabling stable nonlinearities and pooling across matrix-valued and Lie-algebraic data. The ReLN Layer Toolbox (ReLN-Linear, ReLN-ReLU, ReLN-Bracket) provides a complete, symmetry-respecting set of components to build end-to-end networks; experimental results across Platonic solid classification, $sp(4)$ invariants, Lorentz-equivariant jet tagging, and geometry-uncertainty drone state estimation demonstrate strong accuracy, robustness, and practical utility. Overall, ReLNs offer a general, numerically well-conditioned framework for learning with broad linear group symmetries on Lie algebras and matrix-valued data, with potential impact in robotics, particle physics, and computer vision.

Abstract

Encoding symmetries is a powerful inductive bias for improving the generalization of deep neural networks. However, most existing equivariant models are limited to simple symmetries like rotations, failing to address the broader class of general linear transformations, GL(n), that appear in many scientific domains. We introduce Reductive Lie Neurons (ReLNs), a novel neural network architecture exactly equivariant to these general linear symmetries. ReLNs are designed to operate directly on a wide range of structured inputs, including general n-by-n matrices. ReLNs introduce a novel adjoint-invariant bilinear layer to achieve stable equivariance for both Lie-algebraic features and matrix-valued inputs, without requiring redesign for each subgroup. This architecture overcomes the limitations of prior equivariant networks that only apply to compact groups or simple vector data. We validate ReLNs' versatility across a spectrum of tasks: they outperform existing methods on algebraic benchmarks with sl(3) and sp(4) symmetries and achieve competitive results on a Lorentz-equivariant particle physics task. In 3D drone state estimation with geometric uncertainty, ReLNs jointly process velocities and covariances, yielding significant improvements in trajectory accuracy. ReLNs provide a practical and general framework for learning with broad linear group symmetries on Lie algebras and matrix-valued data. Project page: https://reductive-lie-neuron.github.io/

Equivariant Neural Networks for General Linear Symmetries on Lie Algebras

TL;DR

This work tackles learning under broad linear symmetries by introducing Reductive Lie Neurons (ReLNs), which enforce exact adjoint-equivariance for the general linear group and its reductive subalgebras. The core idea is a learnable Ad-invariant bilinear form on the reductive algebra that remains nondegenerate, enabling stable nonlinearities and pooling across matrix-valued and Lie-algebraic data. The ReLN Layer Toolbox (ReLN-Linear, ReLN-ReLU, ReLN-Bracket) provides a complete, symmetry-respecting set of components to build end-to-end networks; experimental results across Platonic solid classification, invariants, Lorentz-equivariant jet tagging, and geometry-uncertainty drone state estimation demonstrate strong accuracy, robustness, and practical utility. Overall, ReLNs offer a general, numerically well-conditioned framework for learning with broad linear group symmetries on Lie algebras and matrix-valued data, with potential impact in robotics, particle physics, and computer vision.

Abstract

Encoding symmetries is a powerful inductive bias for improving the generalization of deep neural networks. However, most existing equivariant models are limited to simple symmetries like rotations, failing to address the broader class of general linear transformations, GL(n), that appear in many scientific domains. We introduce Reductive Lie Neurons (ReLNs), a novel neural network architecture exactly equivariant to these general linear symmetries. ReLNs are designed to operate directly on a wide range of structured inputs, including general n-by-n matrices. ReLNs introduce a novel adjoint-invariant bilinear layer to achieve stable equivariance for both Lie-algebraic features and matrix-valued inputs, without requiring redesign for each subgroup. This architecture overcomes the limitations of prior equivariant networks that only apply to compact groups or simple vector data. We validate ReLNs' versatility across a spectrum of tasks: they outperform existing methods on algebraic benchmarks with sl(3) and sp(4) symmetries and achieve competitive results on a Lorentz-equivariant particle physics task. In 3D drone state estimation with geometric uncertainty, ReLNs jointly process velocities and covariances, yielding significant improvements in trajectory accuracy. ReLNs provide a practical and general framework for learning with broad linear group symmetries on Lie algebras and matrix-valued data. Project page: https://reductive-lie-neuron.github.io/
Paper Structure (67 sections, 9 theorems, 45 equations, 5 figures, 8 tables)

This paper contains 67 sections, 9 theorems, 45 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Reductive Lie Neurons: Architecture
  5. A General $\mathrm{Ad}$-invariant Bilinear Form for Reductive Lie Algebras
  6. Verification and Relation to Prior Bilinear Forms.
  7. The ReLN Layer Toolbox
  8. ReLN-Linear.
  9. Equivariant Nonlinearities.
  10. Equivariant Pooling and Invariant Layers.
  11. Unifying geometric representations.
  12. Experiments
  13. Algebraic Benchmarks on Semisimple Lie Algebras
  14. Platonic Solid Classification
  15. Invariant Function Regression on $\mathfrak{sp}(4)$.
  16. ...and 52 more sections

Key Result

Proposition 4.1

The bilinear form $\widetilde{B}$ is symmetric, $\mathrm{Ad}$-invariant, and nondegenerate. Moreover, $\mathfrak z(\mathfrak g)$ and $[\mathfrak g,\mathfrak g]$ are $\widetilde{B}$-orthogonal, with $\widetilde{B}|_{[\mathfrak g,\mathfrak g]}=B$ and $\widetilde{B}|_{\mathfrak z(\mathfrak g)}=\langle\

Figures (5)

  • Figure 1: Examples of Lie groups and related manifolds in scientific applications. From left: the special linear group $\mathrm{SL}(3)$ (image homography), the Lorentz group $\mathrm{SO}(1,3)$ (spacetime symmetry), symplectic groups $\mathrm{Sp}(n)$ (Hamiltonian mechanics), the $\mathrm{SPD}(3)\oplus\mathbb{R}^{3}$ state space (probabilistic estimation), and the general linear group $\mathrm{GL}(3)$ (modeling stress-strain in continuum mechanics). See Table \ref{['tab:framework_scope']} for a more detailed survey.
  • Figure 2: A taxonomy of selected representative equivariant neural architectures, categorized by the symmetries to which they are equivariant. This diagram situates our work, ReLNs, among other notable methods that are often specialized for subgroups such as $\mathrm{SL}(n)$, $\mathrm{SO}(n)$, or the Euclidean group $\mathrm{E}(n)$. An asterisk ($^*$) denotes methods equivariant to the group’s adjoint action.
  • Figure 3: Adjoint equivariance using a unified representation for diverse geometric inputs. Our framework embeds inputs with different transformation rules, such as velocity ($v \mapsto Rv$) and covariance ($\Sigma \mapsto R\Sigma R^T$), into a common Lie algebra. Therefore, they transform under the same adjoint action $\mathrm{Ad}_g$, with which our network $f$ commutes as shown in the diagram.
  • Figure 4: Qualitative comparison of the best-case (middle), average (left), and challenging (right) test sequences. ReLN models consistently track the ground truth (black) with high fidelity, especially when leveraging covariance. Insets provide a magnified view of the two best-performing variants (ours) to highlight their accuracy.
  • Figure 5: Sample aggressive trajectories generated in the PyBullet simulator.

Theorems & Definitions (27)

  • Definition 4.1: Modified Bilinear Form on a Reductive Lie Algebra
  • Proposition 4.1
  • proof : Proof sketch
  • Definition A.1: Semisimple and Reductive Lie Algebras
  • Example A.1
  • Theorem A.1: Cartan criterion; standard
  • Definition A.2
  • Proposition A.1
  • proof
  • Definition B.1: Reductive decomposition
  • ...and 17 more