DYMAG: Rethinking Message Passing Using Dynamical-systems-based Waveforms

TL;DR

DYMAG is presented, a graph neural network based on a novel form of message aggregation that outperforms baseline models on recovery of graph persistence, generating parameters of random graphs, as well as property prediction for proteins, molecules and materials.

Abstract

We present DYMAG, a graph neural network based on a novel form of message aggregation. Standard message-passing neural networks, which often aggregate local neighbors via mean-aggregation, can be regarded as convolving with a simple rectangular waveform which is non-zero only on 1-hop neighbors of every vertex. Here, we go beyond such local averaging. We will convolve the node features with more sophisticated waveforms generated using dynamics such as the heat equation, wave equation, and the Sprott model (an example of chaotic dynamics). Furthermore, we use snapshots of these dynamics at different time points to create waveforms at many effective scales. Theoretically, we show that these dynamic waveforms can capture salient information about the graph including connected components, connectivity, and cycle structures even with no features. Empirically, we test DYMAG on both real and synthetic benchmarks to establish that DYMAG outperforms baseline models on recovery of graph persistence, generating parameters of random graphs, as well as property prediction for proteins, molecules and materials. Our code is available at https://github.com/KrishnaswamyLab/DYMAG.

Figures (4)

• Figure 1: Visualization of Waveforms (a) Waveforms visualized on a line graph with a signal (feature), where DYMAG provides more diverse waveforms than standard message passing; (b) waveforms and combinations provide low-pass and bandpass filters in the frequency domain.
• Figure 2: Visual illustration of DYMAG (a) Waveform bank creation solving PDEs. (b) Multiscale Aggregation by taking inner product with the waveforms. (c) DYMAG consists of stacked layers and a prediction head.
• Figure 3: Mean squared error (MSE, lower is better) for predicting the generating parameters of random graphs and node classification accuracy (higher is better) for homophilic and heterophilic datasets. (See also \ref{['tab:results-graph-params', 'tab:results-node-classification']} in the Appendix.)
• Figure : WaveformCreation

Theorems & Definitions (31)

• Proposition 3.1: Band-pass information
• proof : Proof sketch
• Proposition 3.2: Identification of Connected Components
• proof : Proof sketch
• Proposition 3.3
• proof : Proof Sketch
• Proposition 3.4: Heat energy
• proof : Proof sketch
• Proposition 3.5: Heat energy between graphs
• proof : Proof sketch