Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Clifford Algebraic Approach to E(n)-Equivariant High-order Graph Neural Networks

Viet-Hoang Tran, Thieu N. Vo, Tho Tran Huu, Tan Minh Nguyen

TL;DR

This paper introduces the Clifford Group Equivariant Graph Neural Networks (CG-EGNNs), a novel EGNN that enhances high-order message passing by integrating high-order local structures in the context of Clifford algebras, and establishes the universality property of the $k$-hop message passing framework.

Abstract

Designing neural network architectures that can handle data symmetry is crucial. This is especially important for geometric graphs whose properties are equivariance under Euclidean transformations. Current equivariant graph neural networks (EGNNs), particularly those using message passing, have a limitation in expressive power. Recent high-order graph neural networks can overcome this limitation, yet they lack equivariance properties, representing a notable drawback in certain applications in chemistry and physical sciences. In this paper, we introduce the Clifford Group Equivariant Graph Neural Networks (CG-EGNNs), a novel EGNN that enhances high-order message passing by integrating high-order local structures in the context of Clifford algebras. As a key benefit of using Clifford algebras, CG-EGNN can learn functions that capture equivariance from positional features. By adopting the high-order message passing mechanism, CG-EGNN gains richer information from neighbors, thus improving model performance. Furthermore, we establish the universality property of the $k$-hop message passing framework, showcasing greater expressive power of CG-EGNNs with additional $k$-hop message passing mechanism. We empirically validate that CG-EGNNs outperform previous methods on various benchmarks including n-body, CMU motion capture, and MD17, highlighting their effectiveness in geometric deep learning.

A Clifford Algebraic Approach to E(n)-Equivariant High-order Graph Neural Networks

TL;DR

This paper introduces the Clifford Group Equivariant Graph Neural Networks (CG-EGNNs), a novel EGNN that enhances high-order message passing by integrating high-order local structures in the context of Clifford algebras, and establishes the universality property of the -hop message passing framework.

Abstract

Designing neural network architectures that can handle data symmetry is crucial. This is especially important for geometric graphs whose properties are equivariance under Euclidean transformations. Current equivariant graph neural networks (EGNNs), particularly those using message passing, have a limitation in expressive power. Recent high-order graph neural networks can overcome this limitation, yet they lack equivariance properties, representing a notable drawback in certain applications in chemistry and physical sciences. In this paper, we introduce the Clifford Group Equivariant Graph Neural Networks (CG-EGNNs), a novel EGNN that enhances high-order message passing by integrating high-order local structures in the context of Clifford algebras. As a key benefit of using Clifford algebras, CG-EGNN can learn functions that capture equivariance from positional features. By adopting the high-order message passing mechanism, CG-EGNN gains richer information from neighbors, thus improving model performance. Furthermore, we establish the universality property of the -hop message passing framework, showcasing greater expressive power of CG-EGNNs with additional -hop message passing mechanism. We empirically validate that CG-EGNNs outperform previous methods on various benchmarks including n-body, CMU motion capture, and MD17, highlighting their effectiveness in geometric deep learning.
Paper Structure (31 sections, 15 theorems, 65 equations, 2 figures, 6 tables)

This paper contains 31 sections, 15 theorems, 65 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Clifford Group Equivariant Graph Neural Networks
  5. Clifford Group Equivariant Graph Neural Networks (CG-EGNNs)
  6. E(n)-equivariant Property of CG-EGNNs
  7. Learning Functions that Capture Positional Equivariance by Clifford Algebras
  8. Adding $k$-hop Message Passing Mechanism to CG-EGNNs
  9. Necessity of Adding the $k$-hop Message Passing to CG-EGNNs
  10. Universality of the $k$-hop message passing mechanism in geometric GNNs
  11. Experimental Results
  12. Concluding Remarks
  13. Introduction of Clifford Group and Clifford Group Equivariant Neural Networks
  14. Clifford Algebra
  15. Clifford Group Equivariant Neural Networks (CGENNs)
  16. ...and 16 more sections

Key Result

Proposition 4.2

The following maps are $\operatorname{O}(n)$-equivariant:

Figures (2)

  • Figure 1: Illustration of high-order message passing mechanism (Eqs. \ref{['partialmessage']}-\ref{['featurecompute']}) in CG-EGNN. Here, the feature of node $1$ is updated by computing and aggregating messages of order $d=1,2,3$ from its neighborhood $\mathcal{N}(1) = \{2,3,4\}$.
  • Figure 2: Ball-and-stick model of molecules in MD17. Our method attains lower MSE for molecules with more complex structures.

Theorems & Definitions (28)

  • Remark 4.1
  • Proposition 4.2
  • Remark 4.3
  • Remark 4.4
  • Corollary 4.5
  • Theorem 5.1
  • Remark 5.2: Universality
  • Remark 5.3
  • Definition B.1: Quadratic forms and quadratic vector spaces
  • Definition B.2: Orthogonal basis
  • ...and 18 more