UA-PDFL: A Personalized Approach for Decentralized Federated Learning

TL;DR

This work tackles non-IID data in decentralized federated learning by introducing UA-PDFL, a framework that uses unit representation to estimate client data distributions without public data. It combines client-wise dropout to mitigate overfitting and layer-wise personalization to maintain a universal feature extractor while tailoring local classifiers, enabling adaptive personalization across varying data skew. The authors provide a convergence analysis under standard assumptions and demonstrate through extensive experiments on CIFAR10/100, SVHN, and PathMNIST that UA-PDFL often outperforms state-of-the-art baselines, particularly in highly non-IID settings, with faster convergence and improved accuracy. While DFL incurs higher communication costs, the proposed approach offers practical gains in learning performance and security, highlighting a promising direction for personalization in decentralized settings.

Abstract

Federated learning (FL) is a privacy preserving machine learning paradigm designed to collaboratively learn a global model without data leakage. Specifically, in a typical FL system, the central server solely functions as an coordinator to iteratively aggregate the collected local models trained by each client, potentially introducing single-point transmission bottleneck and security threats. To mitigate this issue, decentralized federated learning (DFL) has been proposed, where all participating clients engage in peer-to-peer communication without a central server. Nonetheless, DFL still suffers from training degradation as FL does due to the non-independent and identically distributed (non-IID) nature of client data. And incorporating personalization layers into DFL may be the most effective solutions to alleviate the side effects caused by non-IID data. Therefore, in this paper, we propose a novel unit representation aided personalized decentralized federated learning framework, named UA-PDFL, to deal with the non-IID challenge in DFL. By adaptively adjusting the level of personalization layers through the guidance of the unit representation, UA-PDFL is able to address the varying degrees of data skew. Based on this scheme, client-wise dropout and layer-wise personalization are proposed to further enhance the learning performance of DFL. Extensive experiments empirically prove the effectiveness of our proposed method.

Figures (13)

• Figure 1: A simple example of two different types of FL frameworks. (a) Centralized FL framework containing a parameter server; (b) Decentralized FL framework without a parameter server.
• Figure 2: Sequence diagram of DFL. Unlike traditional FL, the communication order could be arbitrary.
• Figure 3: An example of FL with personalization layers.
• Figure 4: The relationship between local data distribution and divergence metric. Firstly, the neural network model is trained by local dataset. And then, the unit representation and divergence metric are obtained by Eq. \ref{['eq:std_rep']} and Eq. \ref{['eq:divergence_metric']}, respectively.
• Figure 5: A simple example of client-wise dropout. (a) All the clients are connected for model aggregation. (b) Only one client is communicated for model replacement.