Accelerating Storage-Based Training for Graph Neural Networks

TL;DR

This work tackles the data-preparation bottleneck in storage-based GNN training on web-scale graphs by introducing AGNES, a 3-layer architecture that uses block-wise storage I/O and hyperbatch-based processing to maximize I/O bandwidth. By co-locating data accesses through a graph-aware data layout and employing asynchronous I/O, AGNES significantly reduces the impact of numerous small I/Os while maintaining or improving cache effectiveness. Empirical results on five real-world graphs show AGNES achieving up to 4.1× faster training and up to 4.5× higher I/O bandwidth utilization than state-of-the-art baselines, with accuracy preserved and convergence accelerated. The approach offers a practical, single-machine alternative to distributed training for large-scale GNN workloads, especially under limited memory and hardware resources.

Abstract

Graph neural networks (GNNs) have achieved breakthroughs in various real-world downstream tasks due to their powerful expressiveness. As the scale of real-world graphs has been continuously growing, a storage-based approach to GNN training has been studied, which leverages external storage (e.g., NVMe SSDs) to handle such web-scale graphs on a single machine. Although such storage-based GNN training methods have shown promising potential in large-scale GNN training, we observed that they suffer from a severe bottleneck in data preparation since they overlook a critical challenge: how to handle a large number of small storage I/Os. To address the challenge, in this paper, we propose a novel storage-based GNN training framework, named AGNES, that employs a method of block-wise storage I/O processing to fully utilize the I/O bandwidth of high-performance storage devices. Moreover, to further enhance the efficiency of each storage I/O, AGNES employs a simple yet effective strategy, hyperbatch-based processing based on the characteristics of real-world graphs. Comprehensive experiments on five real-world graphs reveal that AGNES consistently outperforms four state-of-the-art methods, by up to 4.1X faster than the best competitor. Our code is available at https://github.com/Bigdasgit/agnes-kdd26.

Figures (12)

• Figure 1: Overview of storage-based GNN training.
• Figure 2: Breakdown of the execution time of state-of-the-art GNN training methods (Ginex Par22 and GNNDrive Jia24).
• Figure 3: Overview of AGNES with the 3-layer architecture (Storage, In-memory, and Operation layers).
• Figure 4: Cache hit ratio and amount of storage I/Os of Ginex Par22 with varying storage I/O unit sizes.
• Figure 5: Effect of hyperbatch-based processing. It significantly reduces storage I/Os (e.g., 20 I/Os $\rightarrow$ 5 I/Os).