DF Louvain: Fast Incrementally Expanding Approach for Community Detection on Dynamic Graphs

TL;DR

DF Louvain introduces a parallel, incremental framework for dynamic community detection that aggressively marks a small frontier of affected vertices and expands it as needed during the first Louvain pass. By incrementally updating vertex degrees and community edge weights as auxiliary information, it dramatically reduces work compared to reprocessing the full graph. Empirical results show substantial speedups over Static, Naive-dynamic, and Delta-screening approaches on real-world and large-scale graphs, with modularity comparable to baselines in most cases. The approach yields strong scalability with cores and provides practical guidance: use DF Louvain for speed, and DS Louvain if higher modularity is required in some graphs.

Abstract

Community detection is the problem of recognizing natural divisions in networks. A relevant challenge in this problem is to find communities on rapidly evolving graphs. In this report we present our Parallel Dynamic Frontier (DF) Louvain algorithm, which given a batch update of edge deletions and insertions, incrementally identifies and processes an approximate set of affected vertices in the graph with minimal overhead, while using a novel approach of incrementally updating weighted-degrees of vertices and total edge weights of communities. We also present our parallel implementations of Naive-dynamic (ND) and Delta-screening (DS) Louvain. On a server with a 64-core AMD EPYC-7742 processor, our experiments show that DF Louvain obtains speedups of 179x, 7.2x, and 5.3x on real-world dynamic graphs, compared to Static, ND, and DS Louvain, respectively, and is 183x, 13.8x, and 8.7x faster, respectively, on large graphs with random batch updates. Moreover, DF Louvain improves its performance by 1.6x for every doubling of threads.

Figures (15)

• Figure 1: Comparison of dynamic community detection approaches: Naive-dynamic (ND), Delta-screening (DS), and our Dynamic Frontier (DF) approach. Edge deletions/insertions are indicated with dotted lines. Vertices marked as affected (initially) with each approach are highlighted in yellow, and when entire communities are marked as affected, they are hatched.
• Figure 2: An example explaining our Dynamic Frontier (DF) approach. The community membership of each vertex is shown with border color (red, green, or blue), and the algorithm proceeds from left to right. A batch update arrives, affecting vertices $1$, $2$, $4$, and $12$. In the first iteration, vertex $2$ switches from red to green, impacting neighbors $8$ and $10$. In the second iteration, vertex $4$ changes from red to blue, affecting neighbors $3$, $6$, $12$, $14$, and $15$. Afterward, there are no more community changes.
• Figure 3: A dynamic community detection algorithm $f(.)$ accepts as input the previous graph $G^{t-1}$, community memberships $C^{t-1}$, and the batch update $\Delta^{t-}$, $\Delta^{t+}$, and returns the updated community memberships $C^t$. However, it may also accept weighted degree of vertices $K^{t-1}$ and total edge weights of communities $\Sigma^{t-1}$ as auxiliary information, and generate updated auxiliary information $K^t$, $\Sigma^t$.
• Figure 4: Speedup of Naive-dynamic (ND), Delta-screening (DS), and Dynamic Frontier (DF) Louvain when reusing the previous weighted-degrees of vertices and total edge weight of communities as auxiliary information to the dynamic algorithm, compared to the same dynamic algorithm when both are recomputed from scratch. This is done on large graphs with generated random batch updates of size $10^{-7} |E|$ to $0.1 |E|$.
• Figure 5: Mean Runtime and Modularity of communities obtained with our multicore implementation of Static, Naive-dynamic (ND), Delta-screening (DS), and Dynamic Frontier (DF) Louvain on real-world dynamic graphs, with batch updates of size $10^{-5}|E_T|$ to $10^{-3}|E_T|$. Here, (a) and (b) show the overall runtime and modularity across all temporal graphs, while (c) and (d) show the runtime and modularity for each graph. In (a), the speedup of each approach with respect to Static Louvain is labeled.