Toward Data-centric Directed Graph Learning: An Entropy-driven Approach

TL;DR

This work tackles the data-level limitations of digraph learning by proposing EDEN, a data-centric online knowledge distillation framework that builds a Hierarchical Knowledge Tree (HKT) from directed topology and node profiles and uses entropy-driven mutual information estimation to guide knowledge flow. EDEN enables both a data-centric DiGNN and a model-agnostic hot-and-plug KD module, leveraging multi-perspective discovery, node-adaptive personalized transfer, and leaf-prediction through random walks to produce rich, hierarchical representations. Extensive experiments on 14 digraphs across 4 downstream tasks show state-of-the-art performance and robust improvements over strong baselines, with ablations validating the contributions of diverse knowledge, personalized KD, and KD-based optimization. While demonstrating strong empirical gains, the approach also highlights scalability challenges and motivates future work to simplify the hierarchical data-centric theory for deployment on billion-scale graphs.

Abstract

The directed graph (digraph), as a generalization of undirected graphs, exhibits superior representation capability in modeling complex topology systems and has garnered considerable attention in recent years. Despite the notable efforts made by existing DiGraph Neural Networks (DiGNNs) to leverage directed edges, they still fail to comprehensively delve into the abundant data knowledge concealed in the digraphs. This data-level limitation results in model-level sub-optimal predictive performance and underscores the necessity of further exploring the potential correlations between the directed edges (topology) and node profiles (feature and labels) from a data-centric perspective, thereby empowering model-centric neural networks with stronger encoding capabilities. In this paper, we propose \textbf{E}ntropy-driven \textbf{D}igraph knowl\textbf{E}dge distillatio\textbf{N} (EDEN), which can serve as a data-centric digraph learning paradigm or a model-agnostic hot-and-plug data-centric Knowledge Distillation (KD) module. The core idea is to achieve data-centric ML, guided by our proposed hierarchical encoding theory for structured data. Specifically, EDEN first utilizes directed structural measurements from a topology perspective to construct a coarse-grained Hierarchical Knowledge Tree (HKT). Subsequently, EDEN quantifies the mutual information of node profiles to refine knowledge flow in the HKT, enabling data-centric KD supervision within model training. As a general framework, EDEN can also naturally extend to undirected scenarios and demonstrate satisfactory performance. In our experiments, EDEN has been widely evaluated on 14 (di)graph datasets (homophily and heterophily) and across 4 downstream tasks. The results demonstrate that EDEN attains SOTA performance and exhibits strong improvement for prevalent (Di)GNNs.

Figures (8)

• Figure 1: The overview of our proposed hierarchical encoding theory with HKT layer-wise KD. Its trainable core involves digraph learning function within every HKT layer and MI neural estimation across layers (illustrated using leaf nodes and their parents). Different colors and dotted lines represent distinct labels.
• Figure 2: The overview of our proposed EDEN.
• Figure 3: Efficiency of Node-C on Empire.
• Figure 4: Node-C performance on CoraML.
• Figure 5: The visualization experiments of the interruption issue of single-direction random walk on digraphs. The circle represents a special topology where a node can walk back to itself, and its existence will alleviate walk interruption. The y-axis denotes the proportion of non-walk interruptions.

Theorems & Definitions (4)

• Theorem 3.1
• Theorem 3.2
• Theorem 3.3
• Definition 1.1