Conditional Clifford-Steerable CNNs with Complete Kernel Basis for PDE Modeling

TL;DR

We address the problem of incomplete kernel expressivity in Clifford-Steerable CNNs for PDE modeling under pseudo-Euclidean symmetries. Our approach, Conditional Clifford-Steerable CNNs (C-CSCNNs), conditions kernels on input-derived equivariant representations and employs an implicit parameterization to maintain $O(p,q)$-equivariance (and $E(p,q)$-equivariance of the convolution). We derive the steerability constraint for the conditional kernels and show how it can be solved efficiently. Empirically, C-CSCNNs improve expressivity and achieve state-of-the-art or competitive performance on NS, SWE, MW3, MW2 tasks, with gains in data efficiency and scalability.

Abstract

Clifford-Steerable CNNs (CSCNNs) provide a unified framework that allows incorporating equivariance to arbitrary pseudo-Euclidean groups, including isometries of Euclidean space and Minkowski spacetime. In this work, we demonstrate that the kernel basis of CSCNNs is not complete, thus limiting the model expressivity. To address this issue, we propose Conditional Clifford-Steerable Kernels, which augment the kernels with equivariant representations computed from the input feature field. We derive the equivariance constraint for these input-dependent kernels and show how it can be solved efficiently via implicit parameterization. We empirically demonstrate an improved expressivity of the resulting framework on multiple PDE forecasting tasks, including fluid dynamics and relativistic electrodynamics, where our method consistently outperforms baseline methods.

Figures (7)

• Figure 1: Conditional Clifford-Steerable CNNs use auxiliary information derived from the input feature field to condition the implicit kernel generating $\operatorname{O}(p,q)$-steerable kernels (a). The interaction of the additional features with relative positions remedies limited expressivity of the original approach, yielding richer kernel basis (b) and, consequently, substantial performance gains (c).
• Figure 2: MSE for the Shallow-water equations $\mathbb{R}^{2}$ 1-step forecasting task as a function of the simulations included in the training dataset. Conditional CSCNNs outperform all baselines, keeping their advantage even as the training trajectories increase.
• Figure 3: Mean squared errors for $(1)$ Navier-Stokes $\mathbb{R}^{2}$, $(2)$ Maxwell $\mathbb{R}^{3}$, and $(3)$ relativistic Maxwell $\mathbb{R}^{1,2}$ simulation tasks as a function of the simulations included in the training dataset. Conditional CSCNNs outperform all baselines, with their advantage increasing as more data is included in the training set.
• Figure 4: Relative $L^2$ error of conditioned CSCNNs on the shallow water equations task at different steps of the rollout trajectories. Results are shown for component $u$ of the wind velocity field.
• Figure 5: Illustration of how the receptive field of a finite, discretized kernel changes under rotations. The square support causes operations to break equivariance near the corners.

Theorems & Definitions (15)

• Definition 3.1: Convolution
• Definition 3.2: Equivariance
• Example 3.1
• Theorem 3.1: Steerable Convolution, weiler2023equivariant
• Example 3.2
• Definition 4.1: Conditional Convolution
• Lemma 4.1: Steerable Conditional Convolution
• Lemma 4.2: Equivariance of conditional Clifford-steerable kernels
• Corollary 4.1: Equivariance of Clifford-steerable conditional convolution
• Proposition 4.1