Rademacher Meets Colors: More Expressivity, but at What Cost ?

TL;DR

This work analyzes why increasing GNN expressivity can harm generalization by tying expressivity to color-based partitions via WL colorings and linking this to data-dependent Rademacher complexity. The main result shows the empirical Rademacher complexity is bounded by the number of color classes $p$ induced by the coloring, with tighter bounds to $\sqrt{p/m}$ when outputs lie in $[-1,1]$, demonstrating a concrete expressivity-generalization trade-off. It further proves stability under color-count perturbations and extends the analysis to arbitrary coloring schemes beyond 1-WL, providing a unified framework across GNN architectures. The findings offer principled guidance for designing expressive yet robust GNNs and motivate future work on extending the approach to broader graph-learning tasks and distributional settings.

Abstract

The expressive power of graph neural networks (GNNs) is typically understood through their correspondence with graph isomorphism tests such as the Weisfeiler-Leman (WL) hierarchy. While more expressive GNNs can distinguish a richer set of graphs, they are also observed to suffer from higher generalization error. This work provides a theoretical explanation for this trade-off by linking expressivity and generalization through the lens of coloring algorithms. Specifically, we show that the number of equivalence classes induced by WL colorings directly bounds the GNNs Rademacher complexity -- a key data-dependent measure of generalization. Our analysis reveals that greater expressivity leads to higher complexity and thus weaker generalization guarantees. Furthermore, we prove that the Rademacher complexity is stable under perturbations in the color counts across different samples, ensuring robustness to sampling variability across datasets. Importantly, our framework is not restricted to message-passing GNNs or 1-WL, but extends to arbitrary GNN architectures and expressivity measures that partition graphs into equivalence classes. These results unify the study of expressivity and generalization in GNNs, providing a principled understanding of why increasing expressive power often comes at the cost of generalization.

Figures (1)

• Figure 1: Two function classes $\mathcal{F}_{k\text{-WL}}$ and $\mathcal{F}_{\text{WL}}$, constrained respectively by $1$-WL and $k$-WL expressivity, induce different partitions of a sample $S$. The more expressive function class $\mathcal{F}_{k\text{-WL}}$distinguishes more graphs, leading to a finer partition of the sample and a larger number of equivalence classes. Since Rademacher complexity depends on the number of partitions of the input space, the coarser partition induced by $\mathcal{F}_{\text{WL}}$ yields a tighter upper bound on $\mathcal{R}_S$.

Theorems & Definitions (26)

• Remark 2.1
• Lemma 2.2: Mohri2012-nv [Theorem 3.3]
• Proposition 2.3
• Proposition 3.1
• Corollary 3.2
• proof
• Proposition 3.3: Uniform partitioning assumption
• Proposition 3.4
• Proposition 3.5
• Definition B.1: MPGNN