Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

E(n) Equivariant Topological Neural Networks

Claudio Battiloro, Ege Karaismailoğlu, Mauricio Tec, George Dasoulas, Michelle Audirac, Francesca Dominici

TL;DR

E(n)-Equivariant Topological Neural Networks are introduced, which are E(n)-equivariant message-passing networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes, and E(n)-equivariant variants of TDL models can be directly derived from the framework.

Abstract

Graph neural networks excel at modeling pairwise interactions, but they cannot flexibly accommodate higher-order interactions and features. Topological deep learning (TDL) has emerged recently as a promising tool for addressing this issue. TDL enables the principled modeling of arbitrary multi-way, hierarchical higher-order interactions by operating on combinatorial topological spaces, such as simplicial or cell complexes, instead of graphs. However, little is known about how to leverage geometric features such as positions and velocities for TDL. This paper introduces E(n)-Equivariant Topological Neural Networks (ETNNs), which are E(n)-equivariant message-passing networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes. ETNNs incorporate geometric node features while respecting rotation, reflection, and translation equivariance. Moreover, being TDL models, ETNNs are natively ready for settings with heterogeneous interactions. We provide a theoretical analysis to show the improved expressiveness of ETNNs over architectures for geometric graphs. We also show how E(n)-equivariant variants of TDL models can be directly derived from our framework. The broad applicability of ETNNs is demonstrated through two tasks of vastly different scales: i) molecular property prediction on the QM9 benchmark and ii) land-use regression for hyper-local estimation of air pollution with multi-resolution irregular geospatial data. The results indicate that ETNNs are an effective tool for learning from diverse types of richly structured data, as they match or surpass SotA equivariant TDL models with a significantly smaller computational burden, thus highlighting the benefits of a principled geometric inductive bias. Our implementation of ETNNs can be found at https://github.com/NSAPH-Projects/topological-equivariant-networks.

E(n) Equivariant Topological Neural Networks

TL;DR

E(n)-Equivariant Topological Neural Networks are introduced, which are E(n)-equivariant message-passing networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes, and E(n)-equivariant variants of TDL models can be directly derived from the framework.

Abstract

Graph neural networks excel at modeling pairwise interactions, but they cannot flexibly accommodate higher-order interactions and features. Topological deep learning (TDL) has emerged recently as a promising tool for addressing this issue. TDL enables the principled modeling of arbitrary multi-way, hierarchical higher-order interactions by operating on combinatorial topological spaces, such as simplicial or cell complexes, instead of graphs. However, little is known about how to leverage geometric features such as positions and velocities for TDL. This paper introduces E(n)-Equivariant Topological Neural Networks (ETNNs), which are E(n)-equivariant message-passing networks operating on combinatorial complexes, formal objects unifying graphs, hypergraphs, simplicial, path, and cell complexes. ETNNs incorporate geometric node features while respecting rotation, reflection, and translation equivariance. Moreover, being TDL models, ETNNs are natively ready for settings with heterogeneous interactions. We provide a theoretical analysis to show the improved expressiveness of ETNNs over architectures for geometric graphs. We also show how E(n)-equivariant variants of TDL models can be directly derived from our framework. The broad applicability of ETNNs is demonstrated through two tasks of vastly different scales: i) molecular property prediction on the QM9 benchmark and ii) land-use regression for hyper-local estimation of air pollution with multi-resolution irregular geospatial data. The results indicate that ETNNs are an effective tool for learning from diverse types of richly structured data, as they match or surpass SotA equivariant TDL models with a significantly smaller computational burden, thus highlighting the benefits of a principled geometric inductive bias. Our implementation of ETNNs can be found at https://github.com/NSAPH-Projects/topological-equivariant-networks.
Paper Structure (27 sections, 5 theorems, 28 equations, 11 figures, 12 tables)

This paper contains 27 sections, 5 theorems, 28 equations, 11 figures, 12 tables.

Table of Contents

  1. Limitations & Future Directions
  2. Related Works
  3. Combinatorial Topological Spaces as Combinatorial Complexes
  4. Graphs
  5. Simplicial Complexes
  6. Cell Complexes
  7. Hypergraphs
  8. ETNNs with Velocity Type Inputs
  9. Proof of Theorem \ref{['th:equivariance']}
  10. Expressiveness of ETNNs
  11. Complexity Analysis
  12. Additional References on Real-World Combinatorial Topological Spaces
  13. Molecular Data
  14. Geospatial Data
  15. Experimental Setup
  16. ...and 12 more sections

Key Result

Theorem 1

An ETNN layer as in eq:eq_message_passing-eq:eq_update_pos, synthetically denoted as $\{\mathbf{h}_x^{l+1}\}_{x \in \mathcal{X}},\{\mathbf{x}_z^{l+1}\}_{z \in \mathcal{S}}=\mathrm{ETNN}\left(\{\mathbf{h}_x^l\}_{x \in \mathcal{X}},\{\mathbf{x}_z^l\}_{z \in x}\right)$, is $E(n)$ equivariant, that is for all $(\mathbf{O},\mathbf{b}) \in E(n)$.

Figures (11)

  • Figure 1: Overview of the $E(n)$-Equivariant Topological Neural Networks framework. An input geometric graph/point cloud with (possibly) non-geometric features is provided. A combinatorial complex, whose elements are called cells, is constructed from the input geometric graph/point cloud to encode higher-order hierarchical interactions, based on topological or domain-specific considerations. If available, higher-order cell features can be injected. Geometric invariants for the cells (e.g. pairwise distances, Hausdorff distances, volumes, ...) are computed. Finally, $E(n)$ Equivariant Topological Neural Networks update both geometric and non-geometric features to improve a downstream task while respecting rotation, reflection, and translation equivariance.
  • Figure 2: Real-World Combinatorial Complexes.(Left): A molecular CC where 0-cells are atoms, 1-cells are bonds, and 2-cell represent rings and functional groups. (Right): A geospatial CC where 0-cells are grid points, 1-cells are road polylines, and 2-cells are census tract polygons. For visibility, the geospatial CC only shows one road (rank 1) and its incident lower and higher rank cells.
  • Figure 3: Overview of features used in the air pollution downscaling benchmark. The task is to predict the local $\mathrm{PM}_{2.5}$ measured at the 0-cells, which have an approximate resolution of $22 \times 22$ meters. As such, this is a node regression task.
  • Figure 4: A simplicial complex of order 2, comprising of nodes (cells of rank 0), edges (cells of rank 1), and triangles (cells of rank 2).
  • Figure 5: A CW complex of order 2, comprising of nodes (cells of rank 0), edges (cells of rank 1), and cycles (cells of rank 2).
  • ...and 6 more figures

Theorems & Definitions (18)

  • Theorem 1
  • proof
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • ...and 8 more