How Clustering Affects the Convergence of Decentralized Optimization over Networks: A Monte-Carlo-based Approach

TL;DR

This work addresses how network clustering affects convergence of decentralized optimization. It models large-scale systems with scale-free and clustered-scale-free networks and analyzes the impact of the global clustering coefficient $\mathcal{C}$ on convergence through a gradient-tracking dynamics whose rate is linked to the Laplacian spectrum via $\lambda_{2}(\alpha)$; Monte-Carlo simulations are used due to the analytical intractability of the structure-spectral relation. Across synthetic BA and HK networks and real-world graphs, the study finds that higher clustering generally slows convergence when degree distribution and edge counts are held fixed, with low-clustered networks achieving faster decay of the optimality gap. The findings suggest clustering tuning as a practical lever to accelerate distributed learning and optimization in networked systems.

Abstract

Decentralized algorithms have gained substantial interest owing to advancements in cloud computing, Internet of Things (IoT), intelligent transportation networks, and parallel processing over sensor networks. The convergence of such algorithms is directly related to specific properties of the underlying network topology. Specifically, the clustering coefficient is known to affect, for example, the controllability/observability and the epidemic growth over networks. In this work, we study the effects of the clustering coefficient on the convergence rate of networked optimization approaches. In this regard, we model the structure of large-scale distributed systems by random scale-free (SF) and clustered scale-free (CSF) networks and compare the convergence rate by tuning the network clustering coefficient. This is done by keeping other relevant network properties (such as power-law degree distribution, number of links, and average degree) unchanged. Monte-Carlo-based simulations are used to compare the convergence rate over many trials of SF graph topologies. Furthermore, to study the convergence rate over real case studies, we compare the clustering coefficient of some real-world networks with the eigenspectrum of the underlying network (as a measure of convergence rate). The results interestingly show higher convergence rate over low-clustered networks. This is significant as one can improve the learning rate of many existing decentralized machine-learning scenarios by tuning the network clustering.

Figures (7)

• Figure 1: This figure illustrates the triad formation in HK model to increase the network clustering. The newly added node 'e' makes connection to the preferentially attached node 'a' and connections to the $\mathcal{L}_2$ neighbours of 'a' (in this example $\mathcal{L}_2=3$) to make triangles (or triads). This directly increases the clustering.
• Figure 2: This figure shows examples of SF network via BA model (Top) and CSF network via HK model (Bottom). Both network topologies have the same number of links and average node degree. The CSF network contains more triangles (or triads) as compared to the SF network.
• Figure 3: This figure shows the decentralized learning with loss function given by Eq. \ref{['eq_fij_sim']} over different SF and CSF networks respectively modelled by BA and HK methods. It is clear that for the low-clustered SF network, the convergence is faster.
• Figure 4: This figure shows the decentralized optimization of the cost function given by Eq. \ref{['eq_fi_quad']} over different network topologies. Clearly, the convergence over low-clustered networks is faster.
• Figure 5: This figure maps the clustering coefficient versus algebraic connectivity $\lambda_2(\overline{W})$ for the real networks in Table \ref{['tab_case']}. It can be seen that, in general, larger algebraic connectivity is associated with lower clustering and vice versa.