Can You Hear Me Now? A Benchmark for Long-Range Graph Propagation

TL;DR

This paper addresses the difficulty of capturing long-range interactions in GNNs and introduces ECHO, a benchmark combining synthetic long-range tasks and chemically grounded real-world datasets to stress global information propagation. It demonstrates that standard message-passing models struggle with long-range dependencies, while architectures with global attention or non-dissipative dynamics achieve stronger performance, establishing baselines and actionable insights. ECHO comprises five tasks (three synthetic in ECHO-Synth and two chemistry-based in ECHO-Chem) with explicit propagation ranges, sizes, and topologies, accompanied by thorough experimental analyses and runtime considerations. Overall, ECHO sets a new standard for evaluating long-range graph information propagation and lays the groundwork for designing more scalable, physics-informed GNNs for AI-driven science.

Abstract

Effectively capturing long-range interactions remains a fundamental yet unresolved challenge in graph neural network (GNN) research, critical for applications across diverse fields of science. To systematically address this, we introduce ECHO (Evaluating Communication over long HOps), a novel benchmark specifically designed to rigorously assess the capabilities of GNNs in handling very long-range graph propagation. ECHO includes three synthetic graph tasks, namely single-source shortest paths, node eccentricity, and graph diameter, each constructed over diverse and structurally challenging topologies intentionally designed to introduce significant information bottlenecks. ECHO also includes two real-world datasets, ECHO-Charge and ECHO-Energy, which define chemically grounded benchmarks for predicting atomic partial charges and molecular total energies, respectively, with reference computations obtained at the density functional theory (DFT) level. Both tasks inherently depend on capturing complex long-range molecular interactions. Our extensive benchmarking of popular GNN architectures reveals clear performance gaps, emphasizing the difficulty of true long-range propagation and highlighting design choices capable of overcoming inherent limitations. ECHO thereby sets a new standard for evaluating long-range information propagation, also providing a compelling example for its need in AI for science.

Figures (13)

• Figure 1: Visualization of the proposed topologies in the synthetic dataset. In all graphs, $N=30$.
• Figure 2: The 3D molecular graph of caffeine annotated with atomic partial charges. Blue indicates regions of negative partial charge, while red corresponds to positive charge accumulation.
• Figure 3: Visualization of prediction errors for the ECHO-Charge task using two different GNN architectures: A-DGN (a) and GCN (b). The coloring represents the logarithm of the absolute prediction error, $\log(\lvert y_\text{true} - y_\text{pred} \rvert)$. Lower values (in green) indicate better prediction accuracy, while higher values (in orange) correspond to larger errors.
• Figure 4: Statistics of the ECHO-Synth dataset, reporting the distributions of the total number of nodes, total number of edges, average degree, diameter, and node eccentricity.
• Figure 5: Statistics of the ECHO-Charge dataset, reporting the distributions of the total number of nodes, total number of edges, average degree, and diameter.