An Initial Evaluation of Distributed Graph Algorithms using NWGraph and HPX
Karame Mohammadiporshokooh, Panagiotis Syskakis, Hartmut Kaiser
TL;DR
This paper tackles the scalability bottlenecks of distributed graph analytics by integrating the NWGraph library with the HPX asynchronous many-task runtime to overlap computation and communication. It implements distributed BFS and PageRank to evaluate how well asynchronous runtimes handle latency-bound graph workloads, comparing against the Boost Graph Library. The results show that BFS benefits from the HPX model and outperforms PBGL, while PageRank requires further optimization to surpass Boost, highlighting both the potential and the current challenges of this approach. Overall, the work demonstrates the feasibility of HPX-based distributed graph analytics and sets the stage for broader algorithm coverage and benchmarking against established frameworks.
Abstract
Graphs are central to modeling relationships in scientific computing, data analysis, and AI/ML, but their growing scale can exceed the memory and compute capacity of single nodes, requiring distributed solutions. Existing distributed graph framework, however, face fundamental challenges: graph algorithms are latency-bound, suffer from irregular memory access, and often impose synchronization costs that limit scalability and efficiency. In this work, we present a distributed implementation of the NWGraph library integrated with the HPX runtime system. By leveraging HPX's asynchronous many-task model, our approach aims to reduce synchronization overhead, improve load balance, and provide a foundation for distributed graph analytics. We evaluate this approach using two representative algorithms: Breadth-First-Search (BFS) and (PageRank). Our initial results show that BFS achieves better performance than the distributed Boost Graph Library (BGL), while PageRank remains more challenging, with current implementation not yet outperforming BGL. These findings highlight both the promise and the open challenges of applying asynchronous task-based runtimes to graph processing, and point to opportunities for future optimizations and extensions.