Connectivity-Guided Sparsification of 2-FWL GNNs: Preserving Full Expressivity with Improved Efficiency

TL;DR

This work addresses the tension between expressivity and efficiency in higher-order GNNs by focusing on 2-FWL models and introducing Co-Sparsify, a connectivity-guided sparsification that preserves full 2-FWL expressivity. By restricting 3-node interactions to within biconnected components and 2-node interactions to within connected components, and leveraging Tarjan/block-cut decompositions, the approach achieves $O(n+m)$ preprocessing and substantial per-layer gains without sampling or approximation. The authors prove expressivity equivalence to 2-FWL under injective aggregation and demonstrate empirical gains on substructure counting and real-world benchmarks (e.g., ZINC, QM9), achieving state-of-the-art results with reduced resources. They also discuss limitations, notably in long-range tasks, and propose extensions like distance-aware sparsification and adaptive receptive fields to balance generalization and expressivity in scalable GNNs.

Abstract

Higher-order Graph Neural Networks (HOGNNs) based on the 2-FWL test achieve superior expressivity by modeling 2- and 3-node interactions, but at $\mathcal{O}(n^3)$ computational cost. However, this computational burden is typically mitigated by existing efficiency methods at the cost of reduced expressivity. We propose \textbf{Co-Sparsify}, a connectivity-aware sparsification framework that eliminates \emph{provably redundant} computations while preserving full 2-FWL expressive power. Our key insight is that 3-node interactions are expressively necessary only within \emph{biconnected components} -- maximal subgraphs where every pair of nodes lies on a cycle. Outside these components, structural relationships can be fully captured via 2-node message passing or global readout, rendering higher-order modeling unnecessary. Co-Sparsify restricts 2-node message passing to connected components and 3-node interactions to biconnected ones, removing computation without approximation or sampling. We prove that Co-Sparsified GNNs are as expressive as the 2-FWL test. Empirically, on PPGN, Co-Sparsify matches or exceeds accuracy on synthetic substructure counting tasks and achieves state-of-the-art performance on real-world benchmarks (ZINC, QM9). This study demonstrates that high expressivity and scalability are not mutually exclusive: principled, topology-guided sparsification enables powerful, efficient GNNs with theoretical guarantees.

Figures (3)

• Figure 1: Principle of connectivity-aware sparsification. (a) Biconnected case: 3-node interactions needed to distinguish paths. (b) Cut-node case: 2-node interactions suffice; 3-node redundant. (c) Disconnected case: component-level structural properties (e.g., component size) are captured by component-level readout and global structural properties (e.g., component count) are captured by graph-level readout, making explicit 2- or 3-node modeling unnecessary.
• Figure 2: Justification for connectivity-aware sparsification. (a) Distinguishing graphs with different path multiplicities (e.g., $u \to v$ vs. $u \to t \to v$) requires 3-node interactions. (b) Cut node $c$ separates $u$, $t$, and $v$ into distinct components; pairwise interactions $(u,c)$, $(t,c)$, $(v,c)$ suffice to determine the structure. (c) Cut node $c$ separates $t$ from $\{u,v\}$, with $u$ and $v$ connected; structure is captured by $(u,c)$, $(t,c)$, and $(v,c)$. (d) Cut node $t$ separates $u$ and $v$; interactions $(u,t)$ and $(v,t)$ fully determine the graph.
• Figure 3: Co-Sparsified 2-FWL's expressive power. It distinguishes graphs $G_1$ and $G_2$—indistinguishable by 1-WL but distinguishable by 2-FWL—by capturing structural differences in node and pair representations, enabling correct subgraph detection.

Theorems & Definitions (12)

• Lemma 1: Necessity of 3-Node Interactions in Biconnected Components
• proof
• Lemma 2: Redundancy of 3-Node Interactions Across Cut Nodes
• proof
• Lemma 3: Irrelevance of Interactions Between Disconnected Components
• proof
• Theorem 4
• proof
• proof : Detailed Proof of Lemma \ref{['lem:biconnected_necessity']}
• proof : Detailed Proof of Lemma \ref{['lem:cut_redundancy']}