Clifford Neural Layers for PDE Modeling

TL;DR

The paper introduces Clifford neural layers that treat multiple field components as a single multivector object, enabling geometry-aware PDE surrogates. By elevating convolutions and Fourier transforms to Clifford algebra operators, the method captures cross-field relationships between scalars, vectors, bivectors, and beyond. Empirical results across 2D Navier–Stokes, 2D shallow water, and 3D Maxwell problems show improved generalization at similar parameter counts, with rotation-aware variants offering further gains. The work provides a principled inductive bias for neural PDE surrogates and releases code to facilitate future extensions to other domains and architectures.

Abstract

Partial differential equations (PDEs) see widespread use in sciences and engineering to describe simulation of physical processes as scalar and vector fields interacting and coevolving over time. Due to the computationally expensive nature of their standard solution methods, neural PDE surrogates have become an active research topic to accelerate these simulations. However, current methods do not explicitly take into account the relationship between different fields and their internal components, which are often correlated. Viewing the time evolution of such correlated fields through the lens of multivector fields allows us to overcome these limitations. Multivector fields consist of scalar, vector, as well as higher-order components, such as bivectors and trivectors. Their algebraic properties, such as multiplication, addition and other arithmetic operations can be described by Clifford algebras. To our knowledge, this paper presents the first usage of such multivector representations together with Clifford convolutions and Clifford Fourier transforms in the context of deep learning. The resulting Clifford neural layers are universally applicable and will find direct use in the areas of fluid dynamics, weather forecasting, and the modeling of physical systems in general. We empirically evaluate the benefit of Clifford neural layers by replacing convolution and Fourier operations in common neural PDE surrogates by their Clifford counterparts on 2D Navier-Stokes and weather modeling tasks, as well as 3D Maxwell equations. For similar parameter count, Clifford neural layers consistently improve generalization capabilities of the tested neural PDE surrogates. Source code for our PyTorch implementation is available at https://microsoft.github.io/cliffordlayers/.

Figures (17)

• Figure 1: Fields of the Earth's shallow water model. Vector components of the wind velocities (right) are strongly related, i.e. they form a vector field. Additionally, the wind vector field and the scalar pressure field (left) are related since the gradient of the pressure field causes air movement and subsequently influences the wind components. We therefore aim to describe scalar and vector field as one multivector field, which models the dependencies correctly.
• Figure 2: Multivector components of Clifford algebras.
• Figure 3: Antisymmetry of bivector exterior (wedge) product.
• Figure 4: Sketch of Clifford convolution. Multivector input fields are convolved with multivector kernels.
• Figure 5: Sketch of Fourier Neural Operator (FNO) and Clifford Fourier Operator (CFNO) layers. The real valued Fast Fourier transform (RFFT) over real valued scalar input fields $f(x)$ is replaced by the complex Fast Fourier transform (FFT) over the complex valued dual parts ${\bm{v}}(x)$ and ${\bm{s}}(x)$ of multivector fields ${\bm{f}}(x)$. Pointwise multiplication in the Fourier space via complex weight tensor $W$ is replaced by the geometric product in the Clifford Fourier space via multivector weight tensor ${\bm{W}}$. Additionally, the convolution path is replaced by Clifford convolutions with multivector kernels ${\bm{w}}$.

Theorems & Definitions (3)

• proof
• proof
• proof