The GAP Benchmark Suite

TL;DR

The GAP Benchmark Suite addresses the lack of standardized evaluation for graph processing by defining six representative kernels, a diverse set of input graphs, and uniform measurement methodologies. It provides an optimized reference codebase to ground evaluations while remaining platform-agnostic, enabling fair comparisons across research, hardware, and frameworks. The suite includes concrete verification and tooling to ensure correctness and reproducibility, and it targets shared-memory systems with practical, real-world and synthetic graphs. By offering a clear specification and competitive baselines, GAP aims to accelerate reproducible progress in graph processing research and development.

Abstract

We present a graph processing benchmark suite with the goal of helping to standardize graph processing evaluations. Fewer differences between graph processing evaluations will make it easier to compare different research efforts and quantify improvements. The benchmark not only specifies graph kernels, input graphs, and evaluation methodologies, but it also provides optimized baseline implementations. These baseline implementations are representative of state-of-the-art performance, and thus new contributions should outperform them to demonstrate an improvement. The input graphs are sized appropriately for shared memory platforms, but any implementation on any platform that conforms to the benchmark's specifications could be compared. This benchmark suite can be used in a variety of settings. Graph framework developers can demonstrate the generality of their programming model by implementing all of the benchmark's kernels and delivering competitive performance on all of the benchmark's graphs. Algorithm designers can use the input graphs and the baseline implementations to demonstrate their contribution. Platform designers and performance analysts can use the suite as a workload representative of graph processing.