On Oversquashing in Graph Neural Networks Through the Lens of Dynamical Systems
Alessio Gravina, Moshe Eliasof, Claudio Gallicchio, Davide Bacciu, Carola-Bibiane Schönlieb
TL;DR
Oversquashing limits information transfer in MPNNs across long-range graph interactions. The paper reframes this as a non-dissipative dynamical system problem and introduces SWAN, a space-weight antisymmetric DE-GNN, to achieve global and local non-dissipativity and a constant information flow rate. Theoretical analysis shows that SWAN's Jacobians have zero real parts, yielding non-dissipative propagation, while experiments across graph transfer, graph property prediction, and long-range benchmarks demonstrate strong long-range performance with linear computing complexity. This approach provides a principled, scalable mechanism to mitigate oversquashing and broadens the toolbox for long-range graph learning, offering competitive performance without resorting to dense or multi-hop architectures.
Abstract
A common problem in Message-Passing Neural Networks is oversquashing -- the limited ability to facilitate effective information flow between distant nodes. Oversquashing is attributed to the exponential decay in information transmission as node distances increase. This paper introduces a novel perspective to address oversquashing, leveraging dynamical systems properties of global and local non-dissipativity, that enable the maintenance of a constant information flow rate. We present SWAN, a uniquely parameterized GNN model with antisymmetry both in space and weight domains, as a means to obtain non-dissipativity. Our theoretical analysis asserts that by implementing these properties, SWAN offers an enhanced ability to transmit information over extended distances. Empirical evaluations on synthetic and real-world benchmarks that emphasize long-range interactions validate the theoretical understanding of SWAN, and its ability to mitigate oversquashing.