Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties

TL;DR

The paper investigates whether rotationally equivariant convolutions provide tangible gains for molecular property prediction. By comparing distance-based invariant L0Nets to angular-feature-equipped L1Nets within the e3nn framework on QM9, it demonstrates that angular features yield a substantial average reduction in error (about 23%) at fixed model size, while merely increasing depth offers modest gains (~4%). The authors offer physical intuition via dipole moment reasoning and show that L1Nets outperform L0Nets on most targets, though exceptions exist (e.g., dipole moment where non-rotating SchNetPack can excel). The study also highlights the value of gated nonlinearities and provides a comprehensive supplementary material detailing normalization, hyperparameter search, and learning-plots to support reproducibility and design guidance. Overall, rotationally equivariant layers are recommended when angular contributions to vector-like properties are important, with the caveat that benefits vary by target and dataset characteristics.

Abstract

Equivariant neural networks (ENNs) are graph neural networks embedded in $\mathbb{R}^3$ and are well suited for predicting molecular properties. The ENN library e3nn has customizable convolutions, which can be designed to depend only on distances between points, or also on angular features, making them rotationally invariant, or equivariant, respectively. This paper studies the practical value of including angular dependencies for molecular property prediction directly via an ablation study with \texttt{e3nn} and the QM9 data set. We find that, for fixed network depth and parameter count, adding angular features decreased test error by an average of 23%. Meanwhile, increasing network depth decreased test error by only 4% on average, implying that rotationally equivariant layers are comparatively parameter efficient. We present an explanation of the accuracy improvement on the dipole moment, the target which benefited most from the introduction of angular features.

Figures (4)

• Figure 1:
• Figure 2: The magnitude of the total dipole moment depends on the orientation of the constituents, which L0Net convolutions do not consider.
• Figure 3: Illustration of the hyperparameter search with the set of possible architectures on the left, the set of possible output blocks on the top right, and the multi-target, normalized loss function shown in the bottom right.
• Figure 4: Plotted above is the logarithm of the mean absolute error on the validation set versus the logarithm of training epochs for every regression target. The plots contain the training curves for the L1Net, L0Net, L0Net Deep, L0Net Outdeep, and L0Net Both Deep architectures. Just like in the main article, the adam optimizer was employed with standard parameters and an initial learning rate of $6.53 \times 10^{-3}$. The learning rate was decayed given a loss plateau of five epochs to a minimum of $10^{-7}$. The maximum number of training epochs was set at 200 with early stopping patience of 50.