Generalized Simplicial Attention Neural Networks
Claudio Battiloro, Lucia Testa, Lorenzo Giusti, Stefania Sardellitti, Paolo Di Lorenzo, Sergio Barbarossa
TL;DR
The paper introduces Generalized Simplicial Attention Neural Networks (GSANs), a framework for learning on data defined over simplicial complexes using masked self-attention guided by the Dirac operator $\mathbf{D}_{\mathcal{X}}$. By jointly processing data across multiple simplex orders and enforcing a Dirac-based weight sharing, GSANs provide permutation-equivariant and simplicial-aware learned filters that incorporate harmonic content via a sparse projection, with two variants (GSAN-Joint and GSAN-Hodge) that trade off complexity and topological fidelity. The authors establish theoretical guarantees and demonstrate strong empirical performance across trajectory prediction, missing data imputation, graph classification, and simplex prediction, outperforming state-of-the-art simplicial and graph models. These results underscore the practical impact of topology-aware attention for modeling multi-way interactions in complex systems, with potential extensions in stability, expressivity, and transferability across domains.
Abstract
Graph machine learning methods excel at leveraging pairwise relations present in the data. However, graphs are unable to fully capture the multi-way interactions inherent in many complex systems. An effective way to incorporate them is to model the data on higher-order combinatorial topological spaces, such as Simplicial Complexes (SCs) or Cell Complexes. For this reason, we introduce Generalized Simplicial Attention Neural Networks (GSANs), novel neural network architectures designed to process data living on simplicial complexes using masked self-attentional layers. Hinging on topological signal processing principles, we devise a series of principled self-attention mechanisms able to process data associated with simplices of various order, such as nodes, edges, triangles, and beyond. These schemes learn how to combine data associated with neighbor simplices of consecutive order in a task-oriented fashion, leveraging on the simplicial Dirac operator and its Dirac decomposition. We also prove that GSAN satisfies two fundamental properties: permutation equivariance and simplicial-awareness. Finally, we illustrate how our approach compares favorably with other simplicial and graph models when applied to several (inductive and transductive) tasks such as trajectory prediction, missing data imputation, graph classification, and simplex prediction.