Graph Wave Networks
Juwei Yue, Haikuo Li, Jiawei Sheng, Yihan Guo, Xinghua Zhang, Chuan Zhou, Tingwen Liu, Li Guo
TL;DR
This paper reframes message passing in graph neural networks as a wave propagation process by introducing a graph wave equation (GWE) and two implementations, Graph Wave Networks (GWN-sym and GWN-fa). The key idea is to replace heat-diffusion-based updates with a second-order time derivative, yielding a forward-Euler explicit scheme that is constantly stable under proposed Laplacians and allows larger time steps for efficiency. GWNs connect naturally to traditional spectral GNNs and offer improved handling of wave details, mitigating over-smoothing and addressing heterophily through low- and high-frequency components. Empirical results on diverse datasets demonstrate state-of-the-art performance and robust efficiency, supported by stability analyses and ablations across base models and graph types.
Abstract
Dynamics modeling has been introduced as a novel paradigm in message passing (MP) of graph neural networks (GNNs). Existing methods consider MP between nodes as a heat diffusion process, and leverage heat equation to model the temporal evolution of nodes in the embedding space. However, heat equation can hardly depict the wave nature of graph signals in graph signal processing. Besides, heat equation is essentially a partial differential equation (PDE) involving a first partial derivative of time, whose numerical solution usually has low stability, and leads to inefficient model training. In this paper, we would like to depict more wave details in MP, since graph signals are essentially wave signals that can be seen as a superposition of a series of waves in the form of eigenvector. This motivates us to consider MP as a wave propagation process to capture the temporal evolution of wave signals in the space. Based on wave equation in physics, we innovatively develop a graph wave equation to leverage the wave propagation on graphs. In details, we demonstrate that the graph wave equation can be connected to traditional spectral GNNs, facilitating the design of graph wave networks based on various Laplacians and enhancing the performance of the spectral GNNs. Besides, the graph wave equation is particularly a PDE involving a second partial derivative of time, which has stronger stability on graphs than the heat equation that involves a first partial derivative of time. Additionally, we theoretically prove that the numerical solution derived from the graph wave equation are constantly stable, enabling to significantly enhance model efficiency while ensuring its performance. Extensive experiments show that GWNs achieve SOTA and efficient performance on benchmark datasets, and exhibit outstanding performance in addressing challenging graph problems, such as over-smoothing and heterophily.