DCI: A Coordinated Allocation and Filling Workload-Aware Dual-Cache Allocation GNN Inference Acceleration System
Yi Luo, Yaobin Wang, Qi Wang, Yingchen Song, Huan Wu, Qingfeng Wang, Jun Huang
TL;DR
The paper addresses slow GNN inference on large graphs caused by redundant data loads and CPU-GPU transfers during neighbor sampling. It introduces DCI, a workload-aware dual-cache system that partitions GPU memory between node features and adjacency matrices and uses a lightweight cache-filling algorithm to reduce preprocessing and data loading overhead. Empirical results show end-to-end speedups up to $11.26\times$ over DGL and $13.68\times$ over RAIN, with preprocessing time reduced by up to $98.7\%$, and competitive performance against state-of-the-art single-cache systems and DUCATI while dramatically cutting preprocessing overhead. The approach enhances GPU utilization, lowers preprocessing costs, and improves scalability for inference on large real-world graphs, making it practical for industrial deployments.
Abstract
Graph Neural Networks (GNNs) are powerful tools for processing graph-structured data, increasingly used for large-scale real-world graphs via sampling-based inference methods. However, inherent characteristics of neighbor sampling lead to redundant data loading during GNN inference, compounded by inefficient data transfers between host and GPU memory, resulting in slow inference and low resource utilization. Existing methods to accelerate GNN inference face several challenges: (1) low practical GPU memory utilization, (2) overlooking adjacency matrix locality, and (3) long preprocessing time. To address these challenges, we introduce DCI, an efficient workload-aware dual-cache allocation system for GNN inference acceleration. DCI allocates cache capacities for both node features and adjacency matrices based on workload patterns during the pre-sampling phase, leveraging a lightweight cache-filling algorithm to optimize data loading efficiency. Experimental results demonstrate that DCI accelerates sampling and node feature loading, achieving end-to-end inference speedups of 1.18$\times$ to 11.26$\times$ compared to DGL, and 1.14$\times$ to 13.68$\times$ over RAIN, while reducing preprocessing time by 52.8\% to 98.7\%. Additionally, DCI outperforms state-of-the-art single-cache inference systems by achieving speedup of 1.08$\times$ to 1.32$\times$. We also compared DCI with DUCATI's dual-cache population strategy. Our lightweight population algorithm allows DCI to achieve nearly the same inference speed while keeping preprocessing time to less than 20\% of that required by DUCATI.