Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks

TL;DR

The paper addresses the fragmentation and steep learning curve in topological deep learning by presenting a unified, accessible framework for Topological Neural Networks (TNNs) that operate on generalized topological domains. It standardizes notation, introduces a four-step, tensor-diagram–driven message-passing scheme, and recaps architectures across hypergraphs, simplicial, cellular, and combinatorial complexes, along with their symmetries and tasks. Key contributions include a pedagogical synthesis, a GitHub repository with unified equations, and a critical discussion of benchmarking, domain-generalization, and deeper architectures. The work aims to accelerate adoption and innovation in topology-aware learning by making the field more interoperable and easier to extend to real-world problems.

Abstract

The natural world is full of complex systems characterized by intricate relations between their components: from social interactions between individuals in a social network to electrostatic interactions between atoms in a protein. Topological Deep Learning (TDL) provides a comprehensive framework to process and extract knowledge from data associated with these systems, such as predicting the social community to which an individual belongs or predicting whether a protein can be a reasonable target for drug development. TDL has demonstrated theoretical and practical advantages that hold the promise of breaking ground in the applied sciences and beyond. However, the rapid growth of the TDL literature for relational systems has also led to a lack of unification in notation and language across message-passing Topological Neural Network (TNN) architectures. This presents a real obstacle for building upon existing works and for deploying message-passing TNNs to new real-world problems. To address this issue, we provide an accessible introduction to TDL for relational systems, and compare the recently published message-passing TNNs using a unified mathematical and graphical notation. Through an intuitive and critical review of the emerging field of TDL, we extract valuable insights into current challenges and exciting opportunities for future development.

Figures (11)

• Figure 1: Topological Neural Network: Data associated with a complex system are features defined on a data domain, which is preprocessed into a computational domain that encodes interactions between the system's components with neighborhoods. The TNN's layers use message passing to successively update features and yield an output, e.g. a categorical label in classification or a quantitative value in regression. The output represents new knowledge extracted from the input data.
• Figure 2: Domains: Nodes in blue, (hyper)edges in pink, and faces in dark red. Figure adapted from hajij2022higher.
• Figure 3: Examples of Data on Topological Domains. (a) Higher-order interactions in protein networks. (b) Limited molecular representation: rings can only contain three atoms. (c) Triangular mesh of a protein surface. (d) More flexible molecular representation, permitting the representation of any ring-shaped functional group. (e) Flexible mesh which includes arbitrarily shaped faces. (f) Fully flexible molecular representation, permitting the representation of the complex nested hierarchical structure characteristic of molecules and other natural systems. (g) Hierarchical higher-order interactions in protein networks.
• Figure 4: Lifting Topological Domains. (a) A graph is lifted to a hypergraph by adding hyperedges that connect groups of nodes. (b) A graph can be lifted to a cellular complex by adding faces of any shape. (c) Hyperedges can be added to a cellular complex to lift the structure to a combinatorial complex. Figure adopted from hajij2023tdl.
• Figure 5: Features on a Domain. Left: Features onto three cells—$x$, $y$, and $z$. Right: Skeletons for the entire complex: $X^{(0)}$ contains node features, $X^{(1)}$ contains edge features, and so on.