FedPURIN: Programmed Update and Reduced INformation for Sparse Personalized Federated Learning

TL;DR

FedPURIN tackles non-IID personalization and communication bottlenecks in federated learning by introducing a binary masking mechanism per client and formulating parameter selection as a Quadratic Integer Programming problem. This optimization-guided decoupling identifies critical parameters to upload, enabling sparse aggregation where the server combines critical-client updates with a globally aggregated baseline. Empirical results on Fashion-MNIST, CIFAR-10, and CIFAR-100 under Dirichlet non-IID settings show substantial communication reductions ($46\%-73\%$) with competitive accuracy compared to state-of-the-art PFL methods, and ablations reveal that exact gradients and avoiding BN aggregation contribute to performance gains. The work establishes a practical, theory-grounded approach for communication-efficient personalized federated learning, well-suited for edge intelligence with heterogeneous data sources.

Abstract

Personalized Federated Learning (PFL) has emerged as a critical research frontier addressing data heterogeneity issue across distributed clients. Novel model architectures and collaboration mechanisms are engineered to accommodate statistical disparities while producing client-specific models. Parameter decoupling represents a promising paradigm for maintaining model performance in PFL frameworks. However, the communication efficiency of many existing methods remains suboptimal, sustaining substantial communication burdens that impede practical deployment. To bridge this gap, we propose Federated Learning with Programmed Update and Reduced INformation (FedPURIN), a novel framework that strategically identifies critical parameters for transmission through an integer programming formulation. This mathematically grounded strategy is seamlessly integrated into a sparse aggregation scheme, achieving a significant communication reduction while preserving the efficacy. Comprehensive evaluations on standard image classification benchmarks under varied non-IID conditions demonstrate competitive performance relative to state-of-the-art methods, coupled with quantifiable communication reduction through sparse aggregation. The framework establishes a new paradigm for communication-efficient PFL, particularly advantageous for edge intelligence systems operating with heterogeneous data sources.

Figures (6)

• Figure 1: The workflow of FedPURIN algorithm in communication round $t$. There are mainly four steps: (1) Clients conduct local training to determine critical parameter sets, then upload them; (2) Clients with high similarity on the critical parameters masks are grouped to generate the collaborated weights; (3) Server collects critical parameters from every client to form the trivial global model through FedAvg scheme; (4) Server send back the collaborated/trivial weights for critical/non-critical parameters.
• Figure 2: Comparisons under Dirichlet non-IID on Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets. Dashed lines and arrows annotate the accuracy of FedPURIN.
• Figure 3: Accuracy of different BN schemes on CIFAR-10. The solid line represents the average over three runs.
• Figure 4: Accuracy of different $\tau$s on CIFAR-10. The solid line represents the average over three runs, and the shaded area depicts the range of variation (min–max).
• Figure 5: Distribution of parameter activation frequencies across layers for client and global models during training. Plots of clients (numerically labeled) indicate the fraction of FL rounds each parameter was selected as critical, while the server plot (labeled “S”) reports aggregated non-zero counts.