Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

Yam Eitan, Yoav Gelberg, Guy Bar-Shalom, Fabrizio Frasca, Michael Bronstein, Haggai Maron

TL;DR

This work analyzes the expressivity of Higher-Order Message-Passing (HOMP) in Topological Deep Learning (TDL) from a topological perspective, proving that HOMP cannot distinguish CCs by fundamental invariants such as diameter, orientability, planarity, or homology. It introduces Multi-Cellular Networks (MCN) to achieve full expressivity and Scalable MCN (SMCN) to mitigate computational costs, showing that SMCN can outperform HOMP and expressive graph methods on benchmarks designed to test topological learning. The authors provide a topological indistinguishability criterion based on covering spaces, and demonstrate lifting/pooling can create CCs indistinguishable by HOMP yet differing in topology. Empirically, SMCN achieves superior performance on synthetic torus-based benchmarks and real-world lifted Zinc graphs, with improved capabilities in learning topological properties and diameters, while incurring modest runtime overhead. The results underscore the value of explicitly leveraging topological information in higher-order architectures for robust topological and metric reasoning in graphs and complexes.

Abstract

Topological deep learning (TDL) is a rapidly growing field that seeks to leverage topological structure in data and facilitate learning from data supported on topological objects, ranging from molecules to 3D shapes. Most TDL architectures can be unified under the framework of higher-order message-passing (HOMP), which generalizes graph message-passing to higher-order domains. In the first part of the paper, we explore HOMP's expressive power from a topological perspective, demonstrating the framework's inability to capture fundamental topological and metric invariants such as diameter, orientability, planarity, and homology. In addition, we demonstrate HOMP's limitations in fully leveraging lifting and pooling methods on graphs. To the best of our knowledge, this is the first work to study the expressivity of TDL from a \emph{topological} perspective. In the second part of the paper, we develop two new classes of architectures -- multi-cellular networks (MCN) and scalable MCN (SMCN) -- which draw inspiration from expressive GNNs. MCN can reach full expressivity, but scaling it to large data objects can be computationally expansive. Designed as a more scalable alternative, SMCN still mitigates many of HOMP's expressivity limitations. Finally, we create new benchmarks for evaluating models based on their ability to learn topological properties of complexes. We then evaluate SMCN on these benchmarks and on real-world graph datasets, demonstrating improvements over both HOMP baselines and expressive graph methods, highlighting the value of expressively leveraging topological information. Code and data are available at https://github.com/yoavgelberg/SMCN.

Topological Blindspots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity

TL;DR

This work analyzes the expressivity of Higher-Order Message-Passing (HOMP) in Topological Deep Learning (TDL) from a topological perspective, proving that HOMP cannot distinguish CCs by fundamental invariants such as diameter, orientability, planarity, or homology. It introduces Multi-Cellular Networks (MCN) to achieve full expressivity and Scalable MCN (SMCN) to mitigate computational costs, showing that SMCN can outperform HOMP and expressive graph methods on benchmarks designed to test topological learning. The authors provide a topological indistinguishability criterion based on covering spaces, and demonstrate lifting/pooling can create CCs indistinguishable by HOMP yet differing in topology. Empirically, SMCN achieves superior performance on synthetic torus-based benchmarks and real-world lifted Zinc graphs, with improved capabilities in learning topological properties and diameters, while incurring modest runtime overhead. The results underscore the value of explicitly leveraging topological information in higher-order architectures for robust topological and metric reasoning in graphs and complexes.

Abstract

Topological deep learning (TDL) is a rapidly growing field that seeks to leverage topological structure in data and facilitate learning from data supported on topological objects, ranging from molecules to 3D shapes. Most TDL architectures can be unified under the framework of higher-order message-passing (HOMP), which generalizes graph message-passing to higher-order domains. In the first part of the paper, we explore HOMP's expressive power from a topological perspective, demonstrating the framework's inability to capture fundamental topological and metric invariants such as diameter, orientability, planarity, and homology. In addition, we demonstrate HOMP's limitations in fully leveraging lifting and pooling methods on graphs. To the best of our knowledge, this is the first work to study the expressivity of TDL from a \emph{topological} perspective. In the second part of the paper, we develop two new classes of architectures -- multi-cellular networks (MCN) and scalable MCN (SMCN) -- which draw inspiration from expressive GNNs. MCN can reach full expressivity, but scaling it to large data objects can be computationally expansive. Designed as a more scalable alternative, SMCN still mitigates many of HOMP's expressivity limitations. Finally, we create new benchmarks for evaluating models based on their ability to learn topological properties of complexes. We then evaluate SMCN on these benchmarks and on real-world graph datasets, demonstrating improvements over both HOMP baselines and expressive graph methods, highlighting the value of expressively leveraging topological information. Code and data are available at https://github.com/yoavgelberg/SMCN.
Paper Structure (90 sections, 29 theorems, 88 equations, 15 figures, 5 tables)

This paper contains 90 sections, 29 theorems, 88 equations, 15 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Our contributions.
  3. Previous Work
  4. Topological Deep Learning.
  5. Expressive power of GNNs.
  6. Preliminaries
  7. Notation.
  8. Combinatorial complexes.
  9. Neighborhood functions.
  10. Cochain spaces.
  11. Higher-order message-passing.
  12. Tensor diagrams.
  13. Expressivity Limitations of Higher-Order Message-Passing
  14. A topological criterion for HOMP-indistinguishability
  15. Topological and Metric Limitations
  16. ...and 75 more sections

Key Result

Theorem 4.2

Let ${\mathcal{X}}$ and ${\mathcal{X}}'$ be CCs such that $|{\mathcal{X}}_0| = |{\mathcal{X}}'_0|$. If there exists a CC $\tilde{{\mathcal{X}}}$ that covers each of the connected components of both ${\mathcal{X}}$ and ${\mathcal{X}}'$, then for every HOMP model $\mathsf{M}$, $\mathsf{M}({\mathcal{X}

Figures (15)

  • Figure 1: Pairs of HOMP-indistinguishable complexes differing in fundamental metric/topological properties that. In Figure \ref{['fig:diameter']}, tori with different diameters (top $20$, bottom $22$); in Figure \ref{['fig:orientability_and_planarity']}, a Möbius strip and a cylinder differing in both orientability and planarity; in Figure \ref{['fig:homology']}, a torus and a pair of disconnected tori which have different homology groups.
  • Figure 2: HOMP Tensor diagram.
  • Figure 3: $\text{Cyl}_{h, 2 p}$ covers both $\text{Cyl}_{h, p}$ and $\text{Möb}_{h, p}$.
  • Figure 4: Cylinders are planar.
  • Figure 5: Example tensor diagrams for HOMP, MCN, and SMCN. HOMP only uses nodes labeled with standard cochain spaces. MCN adds nodes labeled with multi-cellular cochain spaces and edges labeled with "equiv" updates. SMCN introduces edges labeled with "SCL". Note that the highest order nodes (green and orange) which can only appear in the MCN diagram are replaced in the SMCN diagram through "SCL" updates of lower order nodes (pink and light blue).
  • ...and 10 more figures

Theorems & Definitions (59)

  • Definition 3.1: Combinatorial complex
  • Definition 4.1: CC covering
  • Theorem 4.2: HOMP-indistinguishability criterion
  • Theorem 4.3: Topological blindspots
  • Proposition 4.4
  • Proposition 5.1: MCN is fully expressive
  • Definition 6.1: Augmented Hasse graph
  • Proposition 6.2
  • Proposition 6.3
  • Proposition 6.4
  • ...and 49 more