Communication Optimization for Decentralized Learning atop Bandwidth-limited Edge Networks

TL;DR

This work tackles the challenge of running decentralized federated learning on bandwidth-limited edge networks by jointly optimizing the overlay communication demands and the communication schedule. The authors formulate the problem, decompose it into tractable subproblems, and develop a suite of algorithms including SDP/ICP-based exact methods and SCA-based heuristics to select activated links, weight the overlay, and route multicast updates under category-based underlay constraints estimated via network tomography. Their approach guarantees convergence within a bounded time and achieves substantial reductions in training time (up to around 80%) without sacrificing accuracy, as demonstrated on real network topologies (Roofnet and IAB) and datasets (CIFAR-10 and MNIST) with both large and small models. The results highlight the value of network-aware co-design for DFL and show overlay routing can further improve performance, even under inference errors about the underlay, signaling practical applicability for edge deployments.

Abstract

Decentralized federated learning (DFL) is a promising machine learning paradigm for bringing artificial intelligence (AI) capabilities to the network edge. Running DFL on top of edge networks, however, faces severe performance challenges due to the extensive parameter exchanges between agents. Most existing solutions for these challenges were based on simplistic communication models, which cannot capture the case of learning over a multi-hop bandwidth-limited network. In this work, we address this problem by jointly designing the communication scheme for the overlay network formed by the agents and the mixing matrix that controls the communication demands between the agents. By carefully analyzing the properties of our problem, we cast each design problem into a tractable optimization and develop an efficient algorithm with guaranteed performance. Our evaluations based on real topology and data show that the proposed algorithm can reduce the total training time by over $80\%$ compared to the baseline without sacrificing accuracy, while significantly improving the computational efficiency over the state of the art.

Figures (14)

• Figure 1: Overlay-based DFL.
• Figure 2: Underlay-aware communication schedule optimization (learning agents: $\{A,B,C,D\}$; underlay nodes: $\{h_1,h_2\}$).
• Figure 3: Challenge for in-overlay aggregation (learning agents: $\{A,B,C,D,E\}$; underlay nodes: $\{h_1,h_2\}$).
• Figure 4: Workflow of overall solution.
• Figure 5: Underlay network topologies.

Theorems & Definitions (14)

• Lemma 3.1
• Definition 1: Huang23MobiHoc
• Lemma 3.2
• Theorem 3.3
• Corollary 3.4
• Lemma 3.5
• Lemma 3.6
• Theorem 3.7
• proof : Proof of Lemma \ref{['lem:equal bandwidth allocation']}
• proof : Proof of Lemma \ref{['lem:equal bandwidth allocation - category']}