Analysis of Dirichlet Energies as Over-smoothing Measures

TL;DR

The paper demonstrates that Dirichlet energy as an over-smoothing metric is not universally valid when GNN dynamics align with the normalized Laplacian. It provides a formal spectral analysis showing non-commutativity between $\Delta$ and $\Delta_{\text{norm}}$, leading to topology-dependent energy behavior and potential misinterpretation of smoothing as norm collapse. Through targeted simulations on ENZYMES and the Cora LCC, it shows that normalized-energy decay can signal genuine spectral smoothing even when embedding norms remain nonzero, highlighting the need for topology-consistent metrics. The work advocates for aligning over-smoothing diagnostics with the operator governing the dynamics and cautions against relying on a single metric across different Laplacian definitions, informing future axiomatic formulations and evaluations in GNN analysis.

Abstract

We analyze the distinctions between two functionals often used as over-smoothing measures: the Dirichlet energies induced by the unnormalized graph Laplacian and the normalized graph Laplacian. We demonstrate that the latter fails to satisfy the axiomatic definition of a node-similarity measure proposed by Rusch \textit{et al.} By formalizing fundamental spectral properties of these two definitions, we highlight critical distinctions necessary to select the metric that is spectrally compatible with the GNN architecture, thereby resolving ambiguities in monitoring the dynamics.

Figures (4)

• Figure 1: Evolution of Dirichlet energies in logarithmic scale and the Frobenius norm of embeddings across 50 GCN layers. The experiment is conducted on the first graph from the ENZYMES dataset, a non-regular graph with 37 nodes, an average degree of 4.54, and a degree variance of 0.98.
• Figure 2: Evolution of Dirichlet energies in logarithmic scale and the Frobenius norm of embeddings across GCN layers. The experiment is conducted on Graph 10 from the ENZYMES dataset, a regular graph with $4$ nodes of degree $3$.
• Figure 3: Evolution in symmetric logarithmic scale of Dirichlet energy (normalized and unnormalized) and Frobenius norm of the embeddings across 50 layers of a GCN without weight matrices in all the layers apart the first one. The experiment uses a custom GCN model on the Largest Connected Component (LCC) of the Cora dataset. The rapid decay of the normalized Dirichlet energy, highlights the over-smoothing phenomenon, while the convergence of $\|X^{(k)}\|_F$ to a non-zero value excludes over-shrinking.
• Figure 4: Evolution of the ratio $\mathcal{E}_{\Delta}(X)/\mathcal{E}_{\Delta_{\text{norm}}}(X)$ through the layers of a GCN implementing $X^{(k+1)} = A_{\text{norm}}X^{(k)}W^{(k)}$ for two graphs of the dataset ENZYMES.

Theorems & Definitions (12)

• Definition 1
• Definition 2
• Remark 1
• Theorem 1
• proof
• Proposition 1
• proof
• Remark 2
• Corollary 1
• proof