Clifford-Steerable Convolutional Neural Networks

TL;DR

Clifford-Steerable CNNs address the challenge of constructing equivariant neural networks for fields on pseudo-Euclidean spaces by combining Clifford algebra with implicit, equivariant kernel networks. The main idea is to implement $ ext{O}(p,q)$-steerable kernels through a kernel network $oldsymbol{ m \mathscr{K}}$ and a kernel head $H$ so that the resulting kernel $K=H\circ\boldsymbol{ m \mathscr{K}}$ satisfies the steerability constraint and yields an $ ext{E}(p,q)$-equivariant convolution. The paper develops the theoretical basis (Clifford grades as $ ext{O}(p,q)$-representations, Clifford-group equivariant nets) and demonstrates empirical gains on physics-informed tasks, including fluid dynamics and relativistic electrodynamics, while enabling extension to curved pseudo-Riemannian manifolds. This work provides a unified, scalable framework for processing multivector fields with full spacetime symmetries, offering improved accuracy and sample efficiency over conventional steerable CNNs and non-equivariant Clifford models. The approach has broad implications for PDE surrogates and physics-embedded learning where $ ext{E}(p,q)$-invariance plays a central role.

Abstract

We present Clifford-Steerable Convolutional Neural Networks (CS-CNNs), a novel class of $\mathrm{E}(p, q)$-equivariant CNNs. CS-CNNs process multivector fields on pseudo-Euclidean spaces $\mathbb{R}^{p,q}$. They cover, for instance, $\mathrm{E}(3)$-equivariance on $\mathbb{R}^3$ and Poincaré-equivariance on Minkowski spacetime $\mathbb{R}^{1,3}$. Our approach is based on an implicit parametrization of $\mathrm{O}(p,q)$-steerable kernels via Clifford group equivariant neural networks. We significantly and consistently outperform baseline methods on fluid dynamics as well as relativistic electrodynamics forecasting tasks.

Figures (11)

• Figure 1: CS-CNNs process multivector fields while respecting $\operatorname{E}(p,q)$-equivariance. Shown here is a Lorentz-boost $\operatorname{O}(1,1)$ of electromagnetic data on 1+1-dimensional spacetime $\mathbb{R}^{1,1}$.
• Figure 2: CS-CNNs process multivector fields while respecting $\operatorname{E}(p,q)$-equivariance. Shown here is a Lorentz-boost $\operatorname{O}(1,1)$ of electromagnetic data on 1+1-dimensional spacetime $\mathbb{R}^{1,1}$.
• Figure 3: Examples of pseudo-Euclidean spaces $\mathbb{R}^{2,0}$ and $\mathbb{R}^{1,1}$. Colors depict $\operatorname{O}(p,q)$-orbits, given by sets of all points $v\in\mathbb{R}^{p,q}$ with the same squared distance$\eta^{p,q}(v,v)$ from the origin.
• Figure 4: Left: Multi-vector valued output of the kernel-network $\mathscr{K}$ for ${{c_\mathrm{in}} \mkern-2mu = \mkern-2mu {c_\mathrm{out}} \mkern-2mu = \mkern-2mu 1, (p,\mkern-2muq) \mkern-2mu = \mkern-2mu (1,\mkern-2mu1)}$, and its expansion to a full ${\operatorname{O}(1,\mkern-3mu1)}$-steerable kernel via the kernel head $H$. Right: Commutative diagram of the construction and ${\operatorname{O}(p,\mkern-2muq)}$-equivariance of implicit steerable kernels ${K\mkern-3mu=\mkern-2muH \mkern-2mu\circ\mkern-2mu \mathscr{K}}$, composed from a kernel network $\mathscr{K}$ with ${{c_\mathrm{out}}\!\times\!{c_\mathrm{in}}}$ multivector outputs and the kernel head $H$. The two inner squares show the individual equivariance of $\mathscr{K}$ and $H$, from which the kernels' overall equivariance follows. We abbreviate $\operatorname{Cl}(\mathbb{R}^{p,q})$ by $\operatorname{Cl}$.
• Figure 5: Plots 1 & 2: Mean squared errors (MSEs) on the Navier-Stokes 2D and Maxwell 3D forecasting tasks (one-step loss) as a function of number of training simulations. Plot 3: Convergence (test loss) of our model vs. a basic ResNet on the relativistic Maxwell task. Plot 4: Relative $\operatorname{O}(2)$-equivariance errors of different models. $G$-FNOs fail as they cannot correctly ingest multivector data.

Theorems & Definitions (63)

• Definition 2.1: Pseudo-Euclidean vector space
• Definition 2.2: Standard pseudo-Euclidean vector spaces
• Example 2.3
• Definition 2.4: Translation groups
• Definition 2.5: Pseudo-orthogonal groups
• Example 2.6
• Definition 2.7: Pseudo-Euclidean groups
• Example 2.8
• Definition 2.9: Feature vector field
• Remark 2.10