Automated Heterogeneous Network learning with Non-Recursive Message Passing

TL;DR

AutoGNR addresses two core challenges in heterogeneous information networks: noise from recursive homogeneous message passing and the need for effective type-aware feature learning. It introduces a non-recursive GNN framework that aggregates hop-specific information separately, combined with a differentiable NAS approach that searches a task-dependent, hop-wise heterogeneous path space. The method demonstrates superior performance and scalability on both normal and large-scale HIN datasets, while providing insights into learned heterogeneous paths and their relevance to downstream tasks. This work offers a practical, adaptable framework for automated, type-aware graph representation learning in complex heterogeneous graphs with improved efficiency and interpretability.

Abstract

Heterogeneous information networks (HINs) can be used to model various real-world systems. As HINs consist of multiple types of nodes, edges, and node features, it is nontrivial to directly apply graph neural network (GNN) techniques in heterogeneous cases. There are two remaining major challenges. First, homogeneous message passing in a recursive manner neglects the distinct types of nodes and edges in different hops, leading to unnecessary information mixing. This often results in the incorporation of ``noise'' from uncorrelated intermediate neighbors, thereby degrading performance. Second, feature learning should be handled differently for different types, which is challenging especially when the type sizes are large. To bridge this gap, we develop a novel framework - AutoGNR, to directly utilize and automatically extract effective heterogeneous information. Instead of recursive homogeneous message passing, we introduce a non-recursive message passing mechanism for GNN to mitigate noise from uncorrelated node types in HINs. Furthermore, under the non-recursive framework, we manage to efficiently perform neural architecture search for an optimal GNN structure in a differentiable way, which can automatically define the heterogeneous paths for aggregation. Our tailored search space encompasses more effective candidates while maintaining a tractable size. Experiments show that AutoGNR consistently outperforms state-of-the-art methods on both normal and large scale real-world HIN datasets.

Figures (6)

• Figure 1: An overview of AutoGNR with $K=2$ on an example HIN. (a) A sample heterogeneous information network of DBLP, consisting of three node types (i.e., $A$, $P$, and $C$, denoted with different shapes) and four edge types (i.e., $AP$, $PA$, $CP$, and $PC$). We use different colors to distinguish neighbors in different hops, and here we regard the yellow Author ($A_1$) node as the anchor (target) node. (b) The specific search space for the anchor node in (a). For each hop, it includes not only individual edge-types but also the possible combinations of them. (c) The optimal architecture derived from search, which (in this example) drops out the information of Paper ($P$) neighbors in the first hop, and aggregates the information of both Paper ($P$) and Conference ($C$) nodes in the second hop. (d) Retrain the searched GNN model from scratch for downstream tasks.
• Figure 2: Illustration of non-recursive GNN framework. Different colors denote neighbors in different hops (e.g., light blue for 1-hop and red for 2-hop neighbors), with the yellow node as the anchor.
• Figure 3: Time cost (GPU seconds) of different models on normal-scale datasets (ACM, DBLP, and IMDB). For NAS-based models, we split the entire time cost into two parts, namely the search part (colored with light blue) and the retraining part (colored with dark blue). The results are obtained from averaging over 50 runs for each model.
• Figure 4: Time cost (measured in GPU seconds) of various models on large-scale datasets (DBLP2 and PubMed). For NAS-based models, the total time cost is divided into two parts: the search phase (colored light blue) and the retraining phase (colored dark blue). Models that encountered out-of-memory issues are left blank. The results represent averages over 50 runs for each model.
• Figure 5: Hop-wise selection frequency distribution. The results are obtained by counting over the 50 times search on 100% training set for each dataset. We scale the frequency to the range of [0,1] for each hop in each dataset.