The ParClusterers Benchmark Suite (PCBS): A Fine-Grained Analysis of Scalable Graph Clustering

TL;DR

The ParClusterers Benchmark Suite (PCBS) is introduced---a collection of highly scalable parallel graph clustering algorithms and benchmarking tools that streamline comparing different graph clustering algorithms and implementations that deliver both the state of the art quality and the running time.

Abstract

We introduce the ParClusterers Benchmark Suite (PCBS) -- a collection of highly scalable parallel graph clustering algorithms and benchmarking tools that streamline comparing different graph clustering algorithms and implementations. The benchmark includes clustering algorithms that target a wide range of modern clustering use cases, including community detection, classification, and dense subgraph mining. The benchmark toolkit makes it easy to run and evaluate multiple instances of different clustering algorithms, which can be useful for fine-tuning the performance of clustering on a given task, and for comparing different clustering algorithms based on different metrics of interest, including clustering quality and running time. Using PCBS, we evaluate a broad collection of real-world graph clustering datasets. Somewhat surprisingly, we find that the best quality results are obtained by algorithms that not included in many popular graph clustering toolkits. The PCBS provides a standardized way to evaluate and judge the quality-performance tradeoffs of the active research area of scalable graph clustering algorithms. We believe it will help enable fair, accurate, and nuanced evaluation of graph clustering algorithms in the future.

Figures (24)

• Figure 1: Overview of our PCBS library.
• Figure 2: Slowdown of methods on SNAP graphs with respect to PCBS. Neo4j cannot load orkut and friendster. TigerGraph cannot load friendster. NetworKit failed to run on friendster. "LD" methods are Leiden-based. "LV" methods are Louvain-based. "MO" is a Girvan-Newman implementation. The horizontal dashed line is at slowdown=1.
• Figure 3: Scalability of modularity clustering implementations on lj with a resolution parameter of 1 and a maximum iteration count of 10. '30h' means using all 30-cores with two-way hyperthreading.
• Figure 4: (Top) The Pareto frontier of the precision and recall of the unweighted SNAP graphs. (Bottom) The Pareto frontier of the $F_{0.5}$ and runtime graph for the unweighted SNAP graphs. "ParHAC$\_{\epsilon}$" shows the curve for ParHAC implementation with approximation parameter $\epsilon$.
• Figure 5: (Top) The Pareto frontier of precision and recall for the weighted $k$-nearest neighbor graphs ($k=50$), using PCBS methods. (Bottom) The Pareto frontier of $F_{0.5}$ score and clustering time on $k$-nearest neighbor graphs ($k=50$). The plot for all 4 graphs are in \ref{['sec:snap_full']} (\ref{['fig:pr_weighted_full']}).