From Coupled Oscillators to Graph Neural Networks: Reducing Over-smoothing via a Kuramoto Model-based Approach
Tuan Nguyen, Hirotada Honda, Takashi Sano, Vinh Nguyen, Shugo Nakamura, Tan M. Nguyen
TL;DR
This work reframes over-smoothing in graph neural networks as a synchronization problem by tying GNN dynamics to the Kuramoto model. It introduces KuramotoGNN, a continuous-depth GNN that uses non-identical natural frequencies to avoid phase synchronization and thus prevents feature collapse across layers, while still reaching a stable synchronized state. The authors provide theoretical connections between over-smoothing, synchronization, and stability, and demonstrate empirically that KuramotoGNN is more depth-robust and often more accurate than baselines across multiple node classification benchmarks. The approach offers a principled bridge between dynamical systems and GNN design, with potential extensions to time delays, adaptive coupling, and second-order damping dynamics.
Abstract
We propose the Kuramoto Graph Neural Network (KuramotoGNN), a novel class of continuous-depth graph neural networks (GNNs) that employs the Kuramoto model to mitigate the over-smoothing phenomenon, in which node features in GNNs become indistinguishable as the number of layers increases. The Kuramoto model captures the synchronization behavior of non-linear coupled oscillators. Under the view of coupled oscillators, we first show the connection between Kuramoto model and basic GNN and then over-smoothing phenomenon in GNNs can be interpreted as phase synchronization in Kuramoto model. The KuramotoGNN replaces this phase synchronization with frequency synchronization to prevent the node features from converging into each other while allowing the system to reach a stable synchronized state. We experimentally verify the advantages of the KuramotoGNN over the baseline GNNs and existing methods in reducing over-smoothing on various graph deep learning benchmark tasks.