DeltaGNN: Graph Neural Network with Information Flow Control

TL;DR

DeltaGNN addresses the persistent issues of over-smoothing and over-squashing in graph neural networks by introducing information flow control (IFC) and a novel information flow score (IFS) to dynamically filter edges during training. The method combines a homophilic aggregation with IFC and a heterophilic graph condensation to capture both short-range and long-range interactions, yielding a scalable architecture that generalizes across diverse graphs. The authors provide theoretical insights and extensive experiments on 10 real-world datasets, showing improved accuracy with linear-time edge filtering and without heavy preprocessing. This approach offers a practical path to robust, long-range-aware GNNs suitable for large-scale, heterogeneous graphs.

Abstract

Graph Neural Networks (GNNs) are popular deep learning models designed to process graph-structured data through recursive neighborhood aggregations in the message passing process. When applied to semi-supervised node classification, the message-passing enables GNNs to understand short-range spatial interactions, but also causes them to suffer from over-smoothing and over-squashing. These challenges hinder model expressiveness and prevent the use of deeper models to capture long-range node interactions (LRIs) within the graph. Popular solutions for LRIs detection are either too expensive to process large graphs due to high time complexity or fail to generalize across diverse graph structures. To address these limitations, we propose a mechanism called \emph{information flow control}, which leverages a novel connectivity measure, called \emph{information flow score}, to address over-smoothing and over-squashing with linear computational overhead, supported by theoretical evidence. Finally, to prove the efficacy of our methodology we design DeltaGNN, the first scalable and generalizable approach for detecting long-range and short-range interactions. We benchmark our model across 10 real-world datasets, including graphs with varying sizes, topologies, densities, and homophilic ratios, showing superior performance with limited computational complexity. The implementation of the proposed methods are publicly available at https://github.com/basiralab/DeltaGNN.

Figures (11)

• Figure 1: Illustration of a graph with a bottleneck.
• Figure 2: Overview of a GNN model with information flow control. The figure illustrates how the information flow control mechanism integrates with a standard GNN to filter the graph and increase the mean node score. Simultaneously, the GNN learns to disentangle the node features, ensuring that the output embeddings can be easily classified using a readout layer. The novel components are highlighted in violet. Notably, the mean node score of the processed graph is significantly higher due to the effective edge filtering performed within the GNN layers.
• Figure 3: The DeltaGNN pipeline consists of: (a) a sequential transformation stage that processes both homophilic and heterophilic edge interactions, performing homophily-based interaction-decoupling and dual aggregation to learn short-term and long-term dependencies; and (b) a prediction stage that concatenates the results from the first stage.
• Figure 4: Validation accuracy (red) and specificity (blue) convergence during training epochs for DeltaGNN linear on three datasets with varying homophilic ratios. The dashed line indicates the observed convergence point.
• Figure 5: Illustrations of first delta embeddings, second delta embeddings, and information flow score on a small graph with bottlenecks and heterophilic edges. We set $m = 1$ and $l = 1$ to detect heterophilic bottlenecks, using the Euclidean distance as the distance metric $d$. The samples used to generate the graph are medical images from the MedMNIST Organ-C dataset medmnistv2. As observed from the plots, nodes 4, 9, and 14 can be easily distinguished as they have very low scores.

Theorems & Definitions (9)

• Definition 2.1: Over-smoothing
• Definition 2.2: Homophilic Ratio of a Node
• Definition 2.3: Over-squashing
• Definition 4.1: First Delta Embeddings
• Definition 4.2: Second Delta Embeddings
• Lemma 1
• proof
• Lemma 2
• proof