An Active Diffusion Neural Network for Graphs

TL;DR

This work addresses the over-smoothing limitation of passive diffusion GNNs by introducing ADGNN, an active diffusion framework that injects external information through ego embeddings, boundary detection, and anomaly cues. It derives a closed-form global embedding by solving the infinite-diffusion limit and links it to an energy minimization formulation, ensuring stable, expressive node representations that reflect global graph structure. Empirically, ADGNN achieves superior accuracy and efficiency across node classification tasks on graphs with varying homophily, hyperspectral image pixel classification, and large-scale graphs, outperforming many diffusion-based and transformer-based baselines. The approach offers a scalable, principled way to integrate multi-scale diffusion information, with Neumann-series approximations enabling efficient computation on large graphs.

Abstract

The analogy to heat diffusion has enhanced our understanding of information flow in graphs and inspired the development of Graph Neural Networks (GNNs). However, most diffusion-based GNNs emulate passive heat diffusion, which still suffers from over-smoothing and limits their ability to capture global graph information. Inspired by the heat death of the universe, which posits that energy distribution becomes uniform over time in a closed system, we recognize that, without external input, node representations in a graph converge to identical feature vectors as diffusion progresses. To address this issue, we propose the Active Diffusion-based Graph Neural Network (ADGNN). ADGNN achieves active diffusion by integrating multiple external information sources that dynamically influence the diffusion process, effectively overcoming the over-smoothing problem. Furthermore, our approach realizes true infinite diffusion by directly calculating the closed-form solution of the active diffusion iterative formula. This allows nodes to preserve their unique characteristics while efficiently gaining comprehensive insights into the graph's global structure. We evaluate ADGNN against several state-of-the-art GNN models across various graph tasks. The results demonstrate that ADGNN significantly improves both accuracy and efficiency, highlighting its effectiveness in capturing global graph information and maintaining node distinctiveness.

Figures (3)

• Figure 1: Flowchart of the proposed ADGNN. Initially, ADGNN computes the ego embeddings for nodes, ${\bm X}^{*}$, utilizing their initial features. Through $K$ diffusion iterations, ADGNN achieves the local embeddings of nodes, ${\bm H}^{(K)}$, reflecting the local structural information of the graph. As the diffusion progresses, the energy of the graph decreases. Upon completion of infinite diffusion, the diffusion output, ${\bm H}^{*}$, is obtained, which encapsulates the global information of the graph and serves as the global embeddings for the nodes. At this stage, the energy of the graph also reaches its minimum. Ultimately, by integrating three diffusion scales of node embeddings (i.e. ${\bm H}^{0}$, ${\bm H}^{(K)}$, and ${\bm H}^{*}$), We derive a high-level node embedding for node classification.
• Figure 2: Running time comparison. Seven competitors and our ADGNN run for 500 epochs during training.
• Figure 3: Pixel classification maps and test accuracies (%) obtained by GCN, DIFFormer, and our ADGNN across various diffusion iteration numbers $K$ on a real HSI, Salinas. Ground truth (GT) denotes the ideal pixel classification results. (a) without diffusion, (c)-(f) GCN, (g)-(j) DIFFormer, (k)-(n) our ADGNN.