Almost Equivariance via Lie Algebra Convolutions

TL;DR

This work defines and operationalizes almost equivariance for learning with Lie group actions by introducing Lie algebra convolutions that operate in the Lie algebra $\mathfrak{g}$ and map to the group via a learnable $\mathcal{N}_{\theta}$ and the exponential map. The proposed $\mathfrak{g}$-conv layer enables flexible, near-equivariant modeling that remains well-defined for non-compact Lie groups, addressing limitations of full group convolutions. The authors formalize the theory connecting almost equivariance with isometries and Ulam stability, and provide existence results for almost isometric embeddings close to fully equivariant ones. Empirically, the Almost Equivariant G-CNN matches or surpasses strong baselines across Rot-MNIST, damped pendulum, smoke plume, and jet-flow tasks, while using fewer parameters than some competing models. Overall, the paper offers a principled, scalable approach to incorporating near-symmetries into neural architectures with potential impact on physics-informed learning and geometric deep learning.

Abstract

Recently, the equivariance of models with respect to a group action has become an important topic of research in machine learning. Analysis of the built-in equivariance of existing neural network architectures, as well as the study of building models that explicitly "bake in" equivariance, have become significant research areas in their own right. However, imbuing an architecture with a specific group equivariance imposes a strong prior on the types of data transformations that the model expects to see. While strictly-equivariant models enforce symmetries, real-world data does not always conform to such strict equivariances. In such cases, the prior of strict equivariance can actually prove too strong and cause models to underperform. Therefore, in this work we study a closely related topic, that of almost equivariance. We provide a definition of almost equivariance and give a practical method for encoding almost equivariance in models by appealing to the Lie algebra of a Lie group. Specifically, we define Lie algebra convolutions and demonstrate that they offer several benefits over Lie group convolutions, including being well-defined for non-compact Lie groups having non-surjective exponential map. From there, we demonstrate connections between the notions of equivariance and isometry and those of almost equivariance and almost isometry. We prove two existence theorems, one showing the existence of almost isometries within bounded distance of isometries of a manifold, and another showing the converse for Hilbert spaces. We extend these theorems to prove the existence of almost equivariant manifold embeddings within bounded distance of fully equivariant embedding functions, subject to certain constraints on the group action and the function class. Finally, we demonstrate the validity of our approach by benchmarking against datasets in fully equivariant and almost equivariant settings.

Figures (3)

• Figure 1: We provide a visualization of how actions of the Lie algebra can be used to approximate actions of the corresponding Lie group. The Lie group, $SO(2)$, of two-dimensional rotations, represented here as the circle, $S^1 \subset \mathbb{C}^2$, with its Lie algebra, $\mathfrak{so}(2)$, represented here as the tangent line at the identity $x = 1 \in \mathbb{C}$, is the most easily visualized case. Here, $\theta$ gives the angle of rotation, $\varepsilon$ gives the approximation error arising from working in the Lie algebra, and the top dashed arrow shows how points can be mapped from the Lie algebra onto the Lie group via the exponential map. The function $\Phi: \mathfrak{g} \to G$, indicated here by the green curve, is a learned mapping that can be trained to approximate the exponential map.
• Figure 2: Training Losses and Train/Validation RMSE across Epochs for Pendulum Trajectory Prediction
• Figure :

Theorems & Definitions (42)

• Definition 3.1
• Definition 3.2
• Definition 3.3
• Definition 3.4
• Definition 3.5
• Definition 3.6
• Theorem 3.7: myersSteenrod
• Remark 3.8
• Theorem 3.9
• proof