Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?

TL;DR

This work challenges the notion that high-degree steerable representations are unnecessary in $E(3)$-equivariant GNNs by showing expressivity degeneration on symmetric graphs when output degree is limited. It develops HEGNN, a high-degree extension of EGNN that uses a scalarization-based interaction to incorporate higher-degree steerables efficiently, and proves that sufficient maximal degree $L$ enables recovery of detailed angle information through inner-product messages. Empirical results on symmetric toy graphs, $N$-body dynamics, and MD17 trajectories validate the theory and demonstrate substantial performance gains over scalarization baselines and competitive results against conventional high-degree models. The findings highlight the importance of higher-degree representations for symmetry-aware geometric learning and offer a scalable approach to greatly improve predictive accuracy in scientific domains.

Abstract

Equivariant Graph Neural Networks (GNNs) that incorporate E(3) symmetry have achieved significant success in various scientific applications. As one of the most successful models, EGNN leverages a simple scalarization technique to perform equivariant message passing over only Cartesian vectors (i.e., 1st-degree steerable vectors), enjoying greater efficiency and efficacy compared to equivariant GNNs using higher-degree steerable vectors. This success suggests that higher-degree representations might be unnecessary. In this paper, we disprove this hypothesis by exploring the expressivity of equivariant GNNs on symmetric structures, including $k$-fold rotations and regular polyhedra. We theoretically demonstrate that equivariant GNNs will always degenerate to a zero function if the degree of the output representations is fixed to 1 or other specific values. Based on this theoretical insight, we propose HEGNN, a high-degree version of EGNN to increase the expressivity by incorporating high-degree steerable vectors while maintaining EGNN's efficiency through the scalarization trick. Our extensive experiments demonstrate that HEGNN not only aligns with our theoretical analyses on toy datasets consisting of symmetric structures, but also shows substantial improvements on more complicated datasets such as $N$-body and MD17. Our theoretical findings and empirical results potentially open up new possibilities for the research of equivariant GNNs.

Figures (2)

• Figure 1: Common symmetric graphs. Equivariant GNNs on symmetric graphs will degenerate to a zero function if the degree of their representations is fixed as 1.
• Figure 2: The different architectures of our HEGNN, EGNN satorras2021en and TFN thomas2018tensor. HEGNN exploits the scalarization trick inspired by EGNN to enable steerable features to interact between different degrees, avoiding the high computational cost of using CG tensor products in TFN.

Theorems & Definitions (17)

• Definition 3.1: Symmetric Graph
• Example 3.2: $k$-folds
• Example 3.3: Regular Polygons
• Theorem 3.4
• Theorem 3.5
• Theorem 3.6
• Theorem 4.1
• Theorem A.1: Equivariance/Invariance of HEGNN
• proof
• Theorem A.2: \ref{['theo:group-average']}