Learning From Simplicial Data Based on Random Walks and 1D Convolutions

TL;DR

The paper addresses the computational challenges of higher-order topological learning by introducing SCRaWl, a simplicial complex neural network that uses random walks on simplices coupled with fast 1D convolutions. By adjusting the number and length of random walks and reusing walks across layers, SCRaWl balances expressivity and efficiency, and its expressiveness is provably incomparable to existing message-passing simplicial networks. Empirically, SCRaWl outperforms competing simplicial models on real-world datasets, notably when higher-order interactions are present, such as co-authorship networks and social-contact graphs. This approach offers a scalable, flexible pathway for leveraging higher-order structure in supervised learning tasks with potential applicability to a broad range of complex systems.

Abstract

Triggered by limitations of graph-based deep learning methods in terms of computational expressivity and model flexibility, recent years have seen a surge of interest in computational models that operate on higher-order topological domains such as hypergraphs and simplicial complexes. While the increased expressivity of these models can indeed lead to a better classification performance and a more faithful representation of the underlying system, the computational cost of these higher-order models can increase dramatically. To this end, we here explore a simplicial complex neural network learning architecture based on random walks and fast 1D convolutions (SCRaWl), in which we can adjust the increase in computational cost by varying the length and number of random walks considered while accounting for higher-order relationships. Importantly, due to the random walk-based design, the expressivity of the proposed architecture is provably incomparable to that of existing message-passing simplicial neural networks. We empirically evaluate SCRaWl on real-world datasets and show that it outperforms other simplicial neural networks.

Figures (8)

• Figure 1: Sketch of SCRaWl illustrating the individual steps of the model. We sample a collection of random walks on the simplicial complex and transform them into walk feature matrices. These are then processed by a 1D convolutional network and pooled into updated simplex states.
• Figure 2: Information flow in SCRaWl modules for simplex orders $0$, $1$, and $2$. Using the walks shown in \ref{['figure:sketch']}, we compute the feature matrices $F_0, F_1$, and $F_2$ based on simplex features $f$ and a window size of $s = 4$. The feature matrices are convolved with a 1D CNN while keeping track of the central simplex within the window (values to the left of the convolution matrices). Convolved matrices are pooled into simplices grouped by the central simplices.
• Figure 3: Architecture of a SCRaWl model operating on simplex orders $0$, $1$, and $2$ with $L$ SCRaWl layers and an user-defined output layer. We reuse the same collection of random walks ${\{W_j\}}_{j \in [m]}$ in each SCRaWl module for performance reasons.
• Figure 4: Expressiveness relations of SCRaWl to other neural network models. SCRaWl and MPSN are strict extensions of their graph counterparts CRaWl and message-passing GNNs, respectively, and their expressive power is incomparable.
• Figure 5: Example of a citation simplicial complex. Each vertex represents an author and connections (edges and triangle) indicate co-authorship. The value of the connection is the total number of citations of the papers the authors have written together.

Theorems & Definitions (2)

• Theorem 1
• proof