Position: Topological Deep Learning is the New Frontier for Relational Learning

TL;DR

The paper argues that Topological Deep Learning (TDL) is the next frontier for relational learning by leveraging higher-order topological structures to capture multi-way interactions and manifold regularities. It surveys practical advantages, showcases compelling examples, and outlines open problems across applications, datasets, software, complexity, explainability, fairness, and theory, proposing concrete directions for progress. Key contributions include a structured problem taxonomy, emphasis on higher-order data and benchmarks, and a roadmap for theoretical and architectural development, including topological representation learning and topological transformers. The work aims to mobilize the research community toward advancing TDL, with broad potential impact across chemistry, biology, NLP, vision, and beyond, by enabling more expressive, robust, and scalable models that respect the topology of complex relational data.

Abstract

Topological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper posits that TDL is the new frontier for relational learning. TDL may complement graph representation learning and geometric deep learning by incorporating topological concepts, and can thus provide a natural choice for various machine learning settings. To this end, this paper discusses open problems in TDL, ranging from practical benefits to theoretical foundations. For each problem, it outlines potential solutions and future research opportunities. At the same time, this paper serves as an invitation to the scientific community to actively participate in TDL research to unlock the potential of this emerging field.

Figures (1)

• Figure 1: Topological spaces enrich deep learning methods from a variety of perspectives. (a): A topological space, modeled as a cell complex, enables a flexible molecular representation. This representation can improve the performance of a deep learning model supported on this space bodnar2021weisfeiler. (b): Topological neural networks enable the processing of data, for example via higher-order message-passing schemes on a topological space. These networks find a wide range of applications, from computer graphics to drug discovery hajij2023tdl. (c): Topological spaces allow hierarchical representations of the underlying data that naturally correspond to pooling operations in deep learning hajij2023tdl. (d): The topological characteristics of the underlying data are crucial when selecting a neural network architecture. Data from a knotted structure in $\mathbb{R}^3$, such as the one shown, cannot be embedded in $\mathbb{R}^2$ with a single layer multilayer perceptron (MLP) from $\mathbb{R}^3$ to $\mathbb{R}^2$olah2014neural.