Position: Don't be Afraid of Over-Smoothing And Over-Squashing

TL;DR

The paper questions the practical impact of over-smoothing and over-squashing in Graph Neural Networks and argues that real-world performance is more affected by uninformative receptive fields and task structure than by these phenomena. It advocates a data/task-driven analysis using statistics that quantify label relevance and information localisation, including Mean Average Distance ($MAD$) and Dirichlet energy ($E$). Across extensive benchmark experiments, the authors show that deeper GNNs and typical mitigation techniques yield limited gains and that optimal depths are modest. The work calls for a paradigm shift in theoretical research toward measuring the localisation and factorisation of label distributions to better align theory with practice.

Abstract

Over-smoothing and over-squashing have been extensively studied in the literature on Graph Neural Networks (GNNs) over the past years. We challenge this prevailing focus in GNN research, arguing that these phenomena are less critical for practical applications than assumed. We suggest that performance decreases often stem from uninformative receptive fields rather than over-smoothing. We support this position with extensive experiments on several standard benchmark datasets, demonstrating that accuracy and over-smoothing are mostly uncorrelated and that optimal model depths remain small even with mitigation techniques, thus highlighting the negligible role of over-smoothing. Similarly, we challenge that over-squashing is always detrimental in practical applications. Instead, we posit that the distribution of relevant information over the graph frequently factorises and is often localised within a small k-hop neighbourhood, questioning the necessity of jointly observing entire receptive fields or engaging in an extensive search for long-range interactions. The results of our experiments show that architectural interventions designed to mitigate over-squashing fail to yield significant performance gains. This position paper advocates for a paradigm shift in theoretical research, urging a diligent analysis of learning tasks and datasets using statistics that measure the underlying distribution of label-relevant information to better understand their localisation and factorisation.

Figures (5)

• Figure 1: MAD with a 3-hop neighbourhood and Dirichlet energy of the whole dataset for all tested models and datasets at different depths after hyperparameter tuning and training. The error bands represent the 25- and 75-percentiles.
• Figure 2: Dirichlet energy and MAD with a 3-hop neighbourhood of the whole dataset for all tested models with GCN message passing and datasets at different depths after hyperparameter tuning and training. The error bands represent the 25- and 75-percentiles.
• Figure 3: Dirichlet energy, MAD with a 3-hop neighbourhood and node similarity by wu2024demystifyingoversmoothingattentionbasedgraph of the whole dataset for all tested models with GATv2 message passing and datasets at different depths after hyperparameter tuning and training. The error bands represent the 25- and 75-percentiles.
• Figure 4: Dirichlet energy, MAD with a 3-hop neighbourhood and node similarity by wu2024demystifyingoversmoothingattentionbasedgraph of the whole dataset for all tested models with GraphSAGE message passing and datasets at different depths after hyperparameter tuning and training. The error bands represent the 25- and 75-percentiles.
• Figure 5: Accuracy (*or average precision) results for rewiring techniques to mitigate over-squashing with corresponding curvature values for the resulting message passing graph.