Breaking Symmetry Bottlenecks in GNN Readouts

TL;DR

This work identifies a fundamental expressivity bottleneck in GNNs at the readout stage: any linear permutation-invariant readout collapses node embeddings to the trivial symmetry component via the Reynolds projection, discarding non-trivial symmetry-aware information regardless of encoder power. To overcome this, the authors propose projector-based invariant readouts that decompose node representations into symmetry-aware channels using equivariant projectors and summarize each channel with nonlinear invariant statistics, thereby preserving permutation invariance while retaining information invisible to averaging. Grounded in representation theory, they establish a factorization result for invariant readouts and demonstrate a practical, graph-dependent two-stage readout that avoids the averaging bottleneck. Empirically, swapping only the readout enables fixed encoders to distinguish WL-hard graph pairs and improves performance on symmetry-heavy benchmarks and several downstream tasks, underscoring the readout as a decisive factor in GNN expressivity.

Abstract

Graph neural networks (GNNs) are widely used for learning on structured data, yet their ability to distinguish non-isomorphic graphs is fundamentally limited. These limitations are usually attributed to message passing; in this work we show that an independent bottleneck arises at the readout stage. Using finite-dimensional representation theory, we prove that all linear permutation-invariant readouts, including sum and mean pooling, factor through the Reynolds (group-averaging) operator and therefore project node embeddings onto the fixed subspace of the permutation action, erasing all non-trivial symmetry-aware components regardless of encoder expressivity. This yields both a new expressivity barrier and an interpretable characterization of what global pooling preserves or destroys. To overcome this collapse, we introduce projector-based invariant readouts that decompose node representations into symmetry-aware channels and summarize them with nonlinear invariant statistics, preserving permutation invariance while retaining information provably invisible to averaging. Empirically, swapping only the readout enables fixed encoders to separate WL-hard graph pairs and improves performance across multiple benchmarks, demonstrating that readout design is a decisive and under-appreciated factor in GNN expressivity.

Figures (4)

• Figure 1: The graphs in Example \ref{['ex:1']}
• Figure 2: Schematic representation of our readout
• Figure 3: Readout scaling on ER graphs ($p=0.1$): Total readout time vs. number of nodes (median and 90th percentile over 50 graphs).
• Figure 4: Left: Complementary Cumulative Distribution Function (CCDF) of per-pair readout time on the 193 measured BREC pairs. Right: Scatter of node count $n$ vs. readout time.

Theorems & Definitions (23)

• Theorem 2.1: Maschke Decomposition, serre_RT, Theorem 2
• Theorem 2.2: Canonical Decomposition, serre_RT
• Definition 3.1: Reynolds Operator, derksen_kemper, p. 38
• Definition 3.2
• Theorem 3.3
• Corollary 3.4
• Example 3.5
• Lemma 4.1
• Lemma 4.2
• Example 4.3