Performance Analysis of Decentralized Federated Learning Deployments

TL;DR

The paper analyzes Decentralized Federated Learning (DFL) as a robust alternative to Centralized Federated Learning, focusing on how topology, non-IID data, and training strategies influence convergence. It introduces a formal global objective and derives convergence-rate bounds for six deployment configurations (Continuous Linear, Continuous Ring, Aggregate Linear, Aggregate Ring, Aggregate Star, Aggregate Mesh) under L-smooth and μ-strong convexity assumptions, highlighting dependence on the non-IID degree Z and gradient noise. The study extends to non-convex models (ResNet, DistilBERT, MiniGPT-4) and conducts extensive experiments across traditional, deep learning, and LLMs, showing IID data yields convergence across deployments, while non-IID data slows convergence and can hinder it for some DL/LLM settings. The results provide actionable guidelines for selecting topology and aggregation strategies in practical DFL deployments within IoT/edge contexts, balancing convergence speed, robustness, and model type. The work thereby bridges theoretical convergence analysis with empirical validation to inform design choices in decentralized, privacy-preserving learning systems.

Abstract

The widespread adoption of smartphones and smart wearable devices has led to the widespread use of Centralized Federated Learning (CFL) for training powerful machine learning models while preserving data privacy. However, CFL faces limitations due to its overreliance on a central server, which impacts latency and system robustness. Decentralized Federated Learning (DFL) is introduced to address these challenges. It facilitates direct collaboration among participating devices without relying on a central server. Each device can independently connect with other devices and share model parameters. This work explores crucial factors influencing the convergence and generalization capacity of DFL models, emphasizing network topologies, non-IID data distribution, and training strategies. We first derive the convergence rate of different DFL model deployment strategies. Then, we comprehensively analyze various network topologies (e.g., linear, ring, star, and mesh) with different degrees of non-IID data and evaluate them over widely adopted machine learning models (e.g., classical, deep neural networks, and Large Language Models) and real-world datasets. The results reveal that models converge to the optimal one for IID data. However, the convergence rate is inversely proportional to the degree of non-IID data distribution. Our findings will serve as valuable guidelines for designing effective DFL model deployments in practical applications.

Figures (1)

• Figure 1: Cross silo vs Cross device