Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization

TL;DR

This work tackles Skewed Heterogeneous Federated Learning (SHFL), where clients exhibit both cross-client label distribution shifts and within-client long-tailed data. It introduces FedPRP, a two-component framework combining Federated Personalization (client-specific adaptive classifiers atop a shared representation) and Federated Prototype Rectification (inter-class discrimination and intra-class consistency losses guided by moving-average prototypes). The method yields personalized local performance and improved global generalization, validated by extensive experiments on CIFAR10, CIFAR100, and Tiny-ImageNet under SHFL partitions, outperforming state-of-the-art baselines and demonstrating strong robustness to skew and new-client scenarios. Overall, FedPRP offers a privacy-preserving approach that aligns local class boundaries with global representation, achieving balanced performance across personalization and generalization in skewed federated settings.

Abstract

Federated learning is an efficient framework designed to facilitate collaborative model training across multiple distributed devices while preserving user data privacy. A significant challenge of federated learning is data-level heterogeneity, i.e., skewed or long-tailed distribution of private data. Although various methods have been proposed to address this challenge, most of them assume that the underlying global data is uniformly distributed across all clients. This paper investigates data-level heterogeneity federated learning with a brief review and redefines a more practical and challenging setting called Skewed Heterogeneous Federated Learning (SHFL). Accordingly, we propose a novel Federated Prototype Rectification with Personalization which consists of two parts: Federated Personalization and Federated Prototype Rectification. The former aims to construct balanced decision boundaries between dominant and minority classes based on private data, while the latter exploits both inter-class discrimination and intra-class consistency to rectify empirical prototypes. Experiments on three popular benchmarks show that the proposed approach outperforms current state-of-the-art methods and achieves balanced performance in both personalization and generalization.

Figures (7)

• Figure 1: Illustration of skewed heterogeneous federated learning. Each participant contains a unique skewed class distribution, while the decision boundary is biased toward the dominant classes. The heterogeneity across participants leads to inconsistencies in the decision boundaries, which further affects the aggregation update of the global model.
• Figure 2: The taxonomy of data heterogeneity of federated learning.
• Figure 3: Illustration of FedPRP. (a) Simplified schematization of our method that solves the SHFL issue via Federated personalization and Federated prototype rectification. (b) Federated personalization: constructing the personalized module $v_k$ that adapts to unique decision boundaries for the class distribution shift. (c) Federated prototype rectification: leveraging Inter-Class Discrimination Loss $\mathcal{L}^{k}_{\mathrm{ID}}$ and Intra-Class Consistency Loss $\mathcal{L}^{k}_{\mathrm{IC}}$ to learn a feature space that brings instances from the same class closer while pushing away those from different classes, ensuring consistency in global optimization to enhance the training of shared representations.
• Figure 4: t-SNE feature visualization of features learned with different methods on $\gamma$ = 0.1 and $s$ = 4. Inconsistent class distribution among different participants, and skewed sample distributions within clients. Features are colored based on classes.
• Figure 5: The $\mathcal{A}^{glo}$ (%) curves on CIFAR10 and CIFAR100 datasets with $\gamma$ = 0.5 under the Sharding non-iid partition strategy.