Personalized Federated Learning Techniques: Empirical Analysis

TL;DR

This work systematically benchmarks ten personalized federated learning (pFL) techniques against two FL baselines to illuminate how personalization interacts with non-iid data, convergence speed, and memory overhead. It classifies pFL methods into three categories based on learning and aggregation strategies, and shows that local-model with personalized aggregation approaches (Category 3) typically converge fastest, albeit with higher per-round costs. Among evaluated methods, FedALA often delivers superior overall performance due to its Adaptive Local Aggregation strategy, while memory-intensive options like FedFomo can incur substantial resource demands. The study provides nuanced insights into accuracy–memory tradeoffs and offers practical guidance for selecting pFL techniques aligned with application constraints and workloads. It also outlines opportunities for further research in convergence analysis, incentive design, and real-world benchmarking.

Abstract

Personalized Federated Learning (pFL) holds immense promise for tailoring machine learning models to individual users while preserving data privacy. However, achieving optimal performance in pFL often requires a careful balancing act between memory overhead costs and model accuracy. This paper delves into the trade-offs inherent in pFL, offering valuable insights for selecting the right algorithms for diverse real-world scenarios. We empirically evaluate ten prominent pFL techniques across various datasets and data splits, uncovering significant differences in their performance. Our study reveals interesting insights into how pFL methods that utilize personalized (local) aggregation exhibit the fastest convergence due to their efficiency in communication and computation. Conversely, fine-tuning methods face limitations in handling data heterogeneity and potential adversarial attacks while multi-objective learning methods achieve higher accuracy at the cost of additional training and resource consumption. Our study emphasizes the critical role of communication efficiency in scaling pFL, demonstrating how it can significantly affect resource usage in real-world deployments.

Figures (3)

• Figure 1: Categorization of PFL algorithms based on learning and aggregation methods. Category 1: Methods that learn a single global model and finetune it. Category 2: Methods that learn additional personalized models. Category 3: Methods that learn local models with personalized (local) aggregation.
• Figure 2: Accuracies of personalized-FL techniques for MNIST and FMNIST datasets.
• Figure 3: Performance of different pFL techniques