Decentralized Personalized Federated Learning
Salma Kharrat, Marco Canini, Samuel Horvath
TL;DR
This work tackles data heterogeneity and communication constraints in decentralized federated learning by introducing DPFL, a bi-level optimization framework that jointly learns personalized models and a directed collaboration graph under a budget B_c. A preprocessing step BGGC constructs initial collaborator pools, while a per-round GGC selects beneficial collaborators through a non-monotone greedy strategy using a reward function R(S) that assesses the combinatorial impact of groups of clients; DPFL then updates local models via decentralized SGD and refines collaborations via alternating minimization. The paper proves approximation guarantees for the graph construction under noisy rewards and demonstrates substantial improvements in average accuracy and variance reduction across CIFAR10, CINIC10, and FEMNIST, with robustness to data flip attacks and ablations showing parameter robustness. The directed collaboration graph enables finer-grained personalization and efficient communication, making the approach scalable to real-world decentralized networks with heterogeneous data distributions. Overall, the proposed framework advances decentralized personalized FL by jointly optimizing models and collaboration graphs under explicit resource constraints and providing empirical evidence of its practical impact.
Abstract
This work tackles the challenges of data heterogeneity and communication limitations in decentralized federated learning. We focus on creating a collaboration graph that guides each client in selecting suitable collaborators for training personalized models that leverage their local data effectively. Our approach addresses these issues through a novel, communication-efficient strategy that enhances resource efficiency. Unlike traditional methods, our formulation identifies collaborators at a granular level by considering combinatorial relations of clients, enhancing personalization while minimizing communication overhead. We achieve this through a bi-level optimization framework that employs a constrained greedy algorithm, resulting in a resource-efficient collaboration graph for personalized learning. Extensive evaluation against various baselines across diverse datasets demonstrates the superiority of our method, named DPFL. DPFL consistently outperforms other approaches, showcasing its effectiveness in handling real-world data heterogeneity, minimizing communication overhead, enhancing resource efficiency, and building personalized models in decentralized federated learning scenarios.