Personalized federated prototype learning in mixed heterogeneous data scenarios

TL;DR

PFPL addresses mixed heterogeneity in federated learning by constructing personalized, domain-aware prototypes for each client and aligning local representations to these prototypes. It introduces cross-domain Lipschitz-constrained prototype aggregation and a local prototype-consistency regularizer, formalized as $\overline{C}_i^{(k)}=\alpha C_i^{(k)}+(1-\alpha)\sum_{m\in M^{(k)}} k_{i,m} C_m^{(k)}$ with $k_{i,m}=\frac{\|C_i^{(k)}-C_m^{(k)}\|_2}{\sum_{m'\in \Phi(k)} \|C_i^{(k)}-C_{m'}^{(k)}\|_2}$ and $\ell_R(\overline{C}_i^{(k)},C_i^{(k)})=\|\overline{C}_i^{(k)}-C_i^{(k)}\|_2}$. The method optimizes $\arg\min_{\phi,\varphi} \sum_{i=1}^m \frac{D_i}{N} \ell_S(f(w_i;x_i),y_i) + \lambda \sum_{k=1}^{|\mathbb{C}|} \sum_{i=1}^m \ell_R(\overline{C}_i^{(k)},C_i^{(k)})$, tying prototype alignment to standard supervised loss. Evaluations on Digits and PACS show higher accuracy and reduced communication cost against baselines under mixed heterogeneity, demonstrating robustness and privacy-preserving cross-domain adaptation. This framework enables scalable personalized federated learning across diverse domains, with potential extensions to CV tasks and adversarially robust prototype learning.

Abstract

Federated learning has received significant attention for its ability to simultaneously protect customer privacy and leverage distributed data from multiple devices for model training. However, conventional approaches often focus on isolated heterogeneous scenarios, resulting in skewed feature distributions or label distributions. Meanwhile, data heterogeneity is actually a key factor in improving model performance. To address this issue, we propose a new approach called PFPL in mixed heterogeneous scenarios. The method provides richer domain knowledge and unbiased convergence targets by constructing personalized, unbiased prototypes for each client. Moreover, in the local update phase, we introduce consistent regularization to align local instances with their personalized prototypes, which significantly improves the convergence of the loss function. Experimental results on Digits and Office Caltech datasets validate the effectiveness of our approach and successfully reduce the communication cost.

Figures (5)

• Figure 1: Architecture of personalized Federated Prototype Learning (PFPL).
• Figure 2: Description of different prototypes.
• Figure 3: t-SNE visualization of the prototype generated by the PFPL method. We consider clients from four different domains in the PACS dataset, corresponding to the $(a),(b),(c)$ and $(d)$ in the picture, and the number of classes for each client is uniformly set to $n = 3$.
• Figure 4: (a) Average test accuracy of PFPL and FedAvg on PACS with varying numbers of samples in each class. (b) Model accuracy corresponding to different $\lambda$. (c) Comparison of the number of parameters transferred in each round of global iteration
• Figure 5: Precision comparison between PFPL and a single heterogeneous personalized federated learning method in mixed heterogeneous scenarios. (a) indicates the skewed feature distribution scenario, and (b) indicates the skewed label distribution scenario.

Theorems & Definitions (2)

• theorem thmcountertheorem
• corollary thmcountercorollary