FedPPA: Progressive Parameter Alignment for Personalized Federated Learning

TL;DR

This work tackles personalization in federated learning under non-IID data and heterogeneous client architectures by introducing FedPPA, which progressively aligns shared layers with the global model during local updates to reduce knowledge inconsistency. FedPPA+ adds entropy-based weighting to the aggregation step, balancing personalization and global performance, with per-client weights $e_i$ derived from data distribution entropy and updates governed by $m^{t+1}_i = igl(e_i(rac{1}{C_l}\sum_{l_i\in l^{t+1}_i} l_i)\bigr)^{L_i}$. Empirical results on MNIST, Fashion-MNIST, and CIFAR-10 show consistent personalization gains over baselines, particularly under strong non-IID conditions, and demonstrate the robustness of the approach across heterogeneous model architectures. Limitations include evaluation restricted to VGG variants and a small-scale setting; future work should test on transformers or larger participant pools to generalize findings. The proposed methods offer a practical path to robust, personalized FL in real-world, resource-diverse environments.

Abstract

Federated Learning (FL) is designed as a decentralized, privacy-preserving machine learning paradigm that enables multiple clients to collaboratively train a model without sharing their data. In real-world scenarios, however, clients often have heterogeneous computational resources and hold non-independent and identically distributed data (non-IID), which poses significant challenges during training. Personalized Federated Learning (PFL) has emerged to address these issues by customizing models for each client based on their unique data distribution. Despite its potential, existing PFL approaches typically overlook the coexistence of model and data heterogeneity arising from clients with diverse computational capabilities. To overcome this limitation, we propose a novel method, called Progressive Parameter Alignment (FedPPA), which progressively aligns the weights of common layers across clients with the global model's weights. Our approach not only mitigates inconsistencies between global and local models during client updates, but also preserves client's local knowledge, thereby enhancing personalization robustness in non-IID settings. To further enhance the global model performance while retaining strong personalization, we also integrate entropy-based weighted averaging into the FedPPA framework. Experiments on three image classification datasets, including MNIST, FMNIST, and CIFAR-10, demonstrate that FedPPA consistently outperforms existing FL algorithms, achieving superior performance in personalized adaptation.

Figures (6)

• Figure 1: Overview of the proposed framework, where common layers are extracted from each client's heterogeneous models. Each client is assigned to an entropy-based contribution weight during global aggregation. The updated model is then returned to the clients, after which parameter alignment is progressively carried out locally based on each client's distinct dataset.
• Figure 2: Distribution of weighted contributions from each client, calculated using Shanon Entropy, applied to different non-IID data with different $\alpha$ values.
• Figure 3: Model accuracy and convergence of our methods in Scenario 1 ($\alpha=0.5$) with two baseline models wangFlexiFedPersonalizedFederated2023mcmahanCommunicationEfficientLearningDeep2023. While all methods can achieve good results, FedPPA and FedPPA+ surpass the baseline performances after 10 training iterations.
• Figure 4: Model accuracy and convergence in Scenario 2 ($\alpha=0.1$). FedPPA and FedPPA+ exhibit stable performance relative to the baselines. As smaller $\alpha$ means more non-IIDness, our methods demonstrated superior performance in personalization.
• Figure 5: Model accuracy and convergence in Scenario 3 ($\alpha=0.01$). As $\alpha$ further decreases, the disparity between our methods and baseline models widens, highlighting the advantages of our proposed methods in handling personalization.