Joint Diffusion Processes as an Inductive Bias in Sheaf Neural Networks
Ferran Hernandez Caralt, Guillermo Bernárdez Gil, Iulia Duta, Pietro Liò, Eduard Alarcón Cot
TL;DR
This paper addresses the failure modes of GNNs under heterophily and oversmoothing by leveraging Sheaf Neural Networks (SNNs), which attach a cellular sheaf to the graph. It introduces two opinion-dynamics–inspired variants (Joint Diffusion and Rotation-Invariant SNNs) that impose a heterophily-friendly inductive bias and reduce parameter counts, along with a dual diffusion mechanism that evolves both node features and restriction maps. A novel synthetic ellipsoid-based benchmark and a controlled Watts–Strogatz–style edge-generation pipeline are proposed to evaluate heterophily handling and data efficiency, with extensive experiments showing competitive performance and insightful behavior under noise and data scarcity. The work highlights practical benefits for scenarios with limited data or feature dimensionality and opens avenues for federated and geometric extensions of SNNs.
Abstract
Sheaf Neural Networks (SNNs) naturally extend Graph Neural Networks (GNNs) by endowing a cellular sheaf over the graph, equipping nodes and edges with vector spaces and defining linear mappings between them. While the attached geometric structure has proven to be useful in analyzing heterophily and oversmoothing, so far the methods by which the sheaf is computed do not always guarantee a good performance in such settings. In this work, drawing inspiration from opinion dynamics concepts, we propose two novel sheaf learning approaches that (i) provide a more intuitive understanding of the involved structure maps, (ii) introduce a useful inductive bias for heterophily and oversmoothing, and (iii) infer the sheaf in a way that does not scale with the number of features, thus using fewer learnable parameters than existing methods. In our evaluation, we show the limitations of the real-world benchmarks used so far on SNNs, and design a new synthetic task -- leveraging the symmetries of n-dimensional ellipsoids -- that enables us to better assess the strengths and weaknesses of sheaf-based models. Our extensive experimentation on these novel datasets reveals valuable insights into the scenarios and contexts where SNNs in general -- and our proposed approaches in particular -- can be beneficial.