Robust Markov stability for community detection at a scale learned based on the structure
Samin Aref, Sanchaai Mathiyarasan
TL;DR
The paper tackles the challenge of robust community detection at an appropriate scale without user-specified hyperparameters by combining multi-scale Markov stability with a pre-trained scale selector. It extends PyGenStability into PyGenStabilityOne (PO), which automatically returns a single partition at a scale $t^*$ learned from graph structure using a gradient-boosted predictor trained on $10^4$ ABCD graphs and Graph2Vec embeddings. PO achieves superior performance relative to 25 of 29 baselines on 500 synthetic ABCD benchmarks and five real networks, demonstrating accuracy, robustness, and hyperparameter-freeness. The work provides practical benefits for CSCW and network analysis by reducing manual tuning and delivering reproducible, structure-informed partitions, with real-world validation and scalability considerations.
Abstract
Community detection, the unsupervised task of clustering nodes of a graph, finds applications across various fields. The common approaches for community detection involve optimizing an objective function to partition the nodes into communities at a single scale of granularity. However, the single-scale approaches often fall short of producing partitions that are robust and at a suitable scale. The existing algorithm, PyGenStability, returns multiple robust partitions for a network by optimizing the multi-scale Markov stability function. However, in cases where the suitable scale is not known or assumed by the user, there is no principled method to select a single robust partition at a suitable scale from the multiple partitions that PyGenStability produces. Our proposed method combines the Markov stability framework with a pre-trained machine learning model for scale selection to obtain one robust partition at a scale that is learned based on the graph structure. This automatic scale selection involves using a gradient boosting model pre-trained on hand-crafted and embedding-based network features from a labeled dataset of 10k benchmark networks. This model was trained to predicts the scale value that maximizes the similarity of the output partition to the planted partition of the benchmark network. Combining our scale selection algorithm with the PyGenStability algorithm results in PyGenStabilityOne (PO): a hyperparameter-free multi-scale community detection algorithm that returns one robust partition at a suitable scale without the need for any assumptions, input, or tweaking from the user. We compare the performance of PO against 29 algorithms and show that it outperforms 25 other algorithms by statistically meaningful margins. Our results facilitate choosing between community detection algorithms, among which PO stands out as the accurate, robust, and hyperparameter-free method.