Efficient Heterogeneous Graph Learning via Random Projection

TL;DR

Heterogeneous Graph Neural Networks often suffer from inefficiency due to repetitive message passing on large graphs. RpHGNN introduces a hybrid pre-computation framework combining representation-wise efficiency with relation-wise low information loss, using propagate-then-update iterations, Random Projection Squashing, and an Even-Odd Propagation Scheme to manage information flow. It achieves state-of-the-art results across seven datasets and is substantially faster (about 230% faster than the best baseline) than strong pre-computation baselines, while also outperforming end-to-end methods on several tasks. This work demonstrates that scalable, accurate HGNNs are attainable by carefully balancing information preservation and computational efficiency, with potential extensions to memory optimization and broader tasks like link prediction.

Abstract

Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for deep learning on heterogeneous graphs. Typical HGNNs require repetitive message passing during training, limiting efficiency for large-scale real-world graphs. Recent pre-computation-based HGNNs use one-time message passing to transform a heterogeneous graph into regular-shaped tensors, enabling efficient mini-batch training. Existing pre-computation-based HGNNs can be mainly categorized into two styles, which differ in how much information loss is allowed and efficiency. We propose a hybrid pre-computation-based HGNN, named Random Projection Heterogeneous Graph Neural Network (RpHGNN), which combines the benefits of one style's efficiency with the low information loss of the other style. To achieve efficiency, the main framework of RpHGNN consists of propagate-then-update iterations, where we introduce a Random Projection Squashing step to ensure that complexity increases only linearly. To achieve low information loss, we introduce a Relation-wise Neighbor Collection component with an Even-odd Propagation Scheme, which aims to collect information from neighbors in a finer-grained way. Experimental results indicate that our approach achieves state-of-the-art results on seven small and large benchmark datasets while also being 230% faster compared to the most effective baseline. Surprisingly, our approach not only surpasses pre-processing-based baselines but also outperforms end-to-end methods.

Figures (12)

• Figure 1: Time consumption ratio of message passing to total for HAN (E2E) and SeHGNN (Pre). (E2E) and (Pre) denote end-to-end and pre-computation-based HGNNs, respectively. HAN (E2E) employs repetitive and resource-intensive message passing during training, constituting a remarkable 95%-99% of the total computation time. Conversely, SeHGNN (Pre) utilizes a one-time message passing approach during pre-computation, leading to a markedly reduced ratio of message passing time relative to total computation time.
• Figure 2: Memory required by HAN for datasets of varying sizes (tested on a GPU with a 24GB memory capacity). HAN is trained end-to-end and relies on the entire graph for message-passing, leading to costly computations for large-scale graphs. As a result, HAN exhibits OOM errors on all large datasets (OGBN-MAG, OAG-Venue, and OAG-L1-Field). Even when processing the significantly smaller dataset, Freebase, the GPU memory is fully consumed.
• Figure 3: Two main styles of pre-computation-based HGNNs.
• Figure 4: Accuracy and time consumption of different HGNNs on a large dataset (OGBN-MAG). (Pre) and (E2E) refer to pre-computation-based and end-to-end HGNNs, respectively.
• Figure 5: An example of neighbors based on a relation (aka meta-path schema). Given a $\mathtt{paper}$ vertex $v_0$, its neighbors based on the relation $\mathtt{paper} \xrightarrow[]{\mathsf{r.write}} \mathtt{author} \xrightarrow[]{\mathsf{write}} \mathtt{paper}$ are $v_1$ and $v_2$. (The two red paths represent meta-paths constrained by the given relation.) This indicates that both papers $v_1$ and $v_2$ are related to paper $v_0$ in terms of sharing an author with it.