From Centralized to Decentralized Federated Learning: Theoretical Insights, Privacy Preservation, and Robustness Challenges
Qiongxiu Li, Wenrui Yu, Yufei Xia, Jun Pang
TL;DR
The paper reframes federated learning by arguing that the core distinction between centralized and decentralized approaches is the training protocol (separate aggregation vs joint optimization) rather than topology alone. It offers a two-axis taxonomy to unify CFL and DFL under a common framework, highlighting how protocol choices influence utility, privacy, and robustness, and identifying a notable gap in fully distributed optimization-based DFL. Through systematic review, it reveals that many so-called decentralized methods rely on aggregation-based heuristics, with underexplored potential for distributed optimization techniques to enhance performance on non-IID data and Byzantine resilience. The work also surveys privacy-attack vectors and defenses (both provable and empirical), discusses debates on CFL vs DFL privacy, and outlines future directions toward protocol-aware designs that balance efficiency, privacy, and robustness in decentralized collaboration.
Abstract
Federated Learning (FL) enables collaborative learning without directly sharing individual's raw data. FL can be implemented in either a centralized (server-based) or decentralized (peer-to-peer) manner. In this survey, we present a novel perspective: the fundamental difference between centralized FL (CFL) and decentralized FL (DFL) is not merely the network topology, but the underlying training protocol: separate aggregation vs. joint optimization. We argue that this distinction in protocol leads to significant differences in model utility, privacy preservation, and robustness to attacks. We systematically review and categorize existing works in both CFL and DFL according to the type of protocol they employ. This taxonomy provides deeper insights into prior research and clarifies how various approaches relate or differ. Through our analysis, we identify key gaps in the literature. In particular, we observe a surprising lack of exploration of DFL approaches based on distributed optimization methods, despite their potential advantages. We highlight this under-explored direction and call for more research on leveraging distributed optimization for federated learning. Overall, this work offers a comprehensive overview from centralized to decentralized FL, sheds new light on the core distinctions between approaches, and outlines open challenges and future directions for the field.