Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Message Passing Neural Networks with Diffusion Distance-guided Stress Majorization

Haoran Zheng, Renchi Yang, Yubo Zhou, Jianliang Xu

TL;DR

This work addresses over-smoothing and feature over-correlation in MPNNs by replacing the Dirichlet-energy objective with a diffusion-distance-guided stress majorization framework and an orthogonal regularization term. The proposed DDSM uses diffusion distances $\Delta(v_i,v_j)$ to set a target inter-node distance $\delta_{i,j}$ and derives a new message passing rule that preserves discriminability on both homophilic and heterophilic graphs, with efficient distance approximation via a truncated spectral decomposition. The approach yields strong empirical gains across 11 datasets and provides theoretical robustness guarantees for diffusion-distance estimates under graph perturbations. Overall, DDSM offers a principled, scalable path to robust graph learning by uniting diffusion geometry with stress-based objectives and feature decorrelation, improving performance in diverse graph structures.

Abstract

Message passing neural networks (MPNNs) have emerged as go-to models for learning on graph-structured data in the past decade. Despite their effectiveness, most of such models still incur severe issues such as over-smoothing and -correlation, due to their underlying objective of minimizing the Dirichlet energy and the derived neighborhood aggregation operations. In this paper, we propose the DDSM, a new MPNN model built on an optimization framework that includes the stress majorization and orthogonal regularization for overcoming the above issues. Further, we introduce the diffusion distances for nodes into the framework to guide the new message passing operations and develop efficient algorithms for distance approximations, both backed by rigorous theoretical analyses. Our comprehensive experiments showcase that DDSM consistently and considerably outperforms 15 strong baselines on both homophilic and heterophilic graphs.

Rethinking Message Passing Neural Networks with Diffusion Distance-guided Stress Majorization

TL;DR

This work addresses over-smoothing and feature over-correlation in MPNNs by replacing the Dirichlet-energy objective with a diffusion-distance-guided stress majorization framework and an orthogonal regularization term. The proposed DDSM uses diffusion distances to set a target inter-node distance and derives a new message passing rule that preserves discriminability on both homophilic and heterophilic graphs, with efficient distance approximation via a truncated spectral decomposition. The approach yields strong empirical gains across 11 datasets and provides theoretical robustness guarantees for diffusion-distance estimates under graph perturbations. Overall, DDSM offers a principled, scalable path to robust graph learning by uniting diffusion geometry with stress-based objectives and feature decorrelation, improving performance in diverse graph structures.

Abstract

Message passing neural networks (MPNNs) have emerged as go-to models for learning on graph-structured data in the past decade. Despite their effectiveness, most of such models still incur severe issues such as over-smoothing and -correlation, due to their underlying objective of minimizing the Dirichlet energy and the derived neighborhood aggregation operations. In this paper, we propose the DDSM, a new MPNN model built on an optimization framework that includes the stress majorization and orthogonal regularization for overcoming the above issues. Further, we introduce the diffusion distances for nodes into the framework to guide the new message passing operations and develop efficient algorithms for distance approximations, both backed by rigorous theoretical analyses. Our comprehensive experiments showcase that DDSM consistently and considerably outperforms 15 strong baselines on both homophilic and heterophilic graphs.

Paper Structure

This paper contains 35 sections, 11 theorems, 22 equations, 6 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Classic Message Passing Neural Networks
  4. Graph Filtering in MPNNs
  5. Diffusion-improved Graph Learning
  6. Background
  7. Diffusion Distance
  8. Message Passing Neural Networks (MPNNs)
  9. Limitation Analyses of MPNNs
  10. Dirichlet Energy Minimization Perspective
  11. Node-wise Issues
  12. Feature-wise Issues
  13. Empirical Studies
  14. Methodology
  15. Design Overview
  16. ...and 20 more sections

Key Result

lemma 1

Let $d_{\min}=\min_{v_k\in\mathcal{V}\xspace}{d_k}$. Then, $\forall{v_i,v_j\in \mathcal{V}\xspace}$, $\Delta_{\textnormal{vanilla}} (v_i, v_j)\in[0,\frac{\sqrt{2}}{d_{\min}}], \Delta_{\textnormal{PR}}(v_i, v_j)\in[0,\frac{\sqrt{2}}{(1-\gamma) d_{\min}}]$ and $\Delta_{\textnormal{HK}}(v_i, v_j)\in[0,

Figures (6)

  • Figure 1: Acc, SMV, Corr, and HOS of $\mathbf{H}\xspace^{(k)}$ in GCN.
  • Figure 2: Illustration of the basic idea of DDSM.
  • Figure 3: Accuracy when varying parameters.
  • Figure 4: Acc, SMV, Corr, and HOS of $\mathbf{H}\xspace^{(k)}$ in GAT.
  • Figure 5: Acc, SMV, Corr, and HOS of $\mathbf{H}\xspace^{(k)}$ in GIN.
  • ...and 1 more figures

Theorems & Definitions (11)

  • lemma 1
  • lemma 2
  • lemma 3
  • lemma 4
  • lemma 5: sun2023feature
  • theorem 1
  • theorem 2
  • theorem 3
  • proposition 1
  • proposition 2
  • ...and 1 more