MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes
Xin-Chun Li, Shaoming Song, Yinchuan Li, Bingshuai Li, Yunfeng Shao, Yang Yang, De-Chuan Zhan
TL;DR
MAP addresses federated learning under incomplete class distributions by unifying two components: Restricted Softmax (RS) to stabilize aggregation against missing classes and Inherited Private Model (HPM) to preserve and leverage historical personalization. The two-stage personalization in MAP—RS-driven initialization followed by standard training with knowledge transfer from HPM—enables simultaneous improvements in both global generalization and local client performance. Theoretical analysis and extensive experiments on multiple datasets demonstrate MAP’s ability to outperform standard FL methods and approach or exceed specialized aggregation or personalization baselines, while also offering scalability to large-scale settings like FEMNIST. This work provides a practical, principled approach to non-I.I.D. FL where clients observe only subsets of classes, with potential implications for fairness and robustness in real-world deployments.
Abstract
In some real-world applications, data samples are usually distributed on local devices, where federated learning (FL) techniques are proposed to coordinate decentralized clients without directly sharing users' private data. FL commonly follows the parameter server architecture and contains multiple personalization and aggregation procedures. The natural data heterogeneity across clients, i.e., Non-I.I.D. data, challenges both the aggregation and personalization goals in FL. In this paper, we focus on a special kind of Non-I.I.D. scene where clients own incomplete classes, i.e., each client can only access a partial set of the whole class set. The server aims to aggregate a complete classification model that could generalize to all classes, while the clients are inclined to improve the performance of distinguishing their observed classes. For better model aggregation, we point out that the standard softmax will encounter several problems caused by missing classes and propose "restricted softmax" as an alternative. For better model personalization, we point out that the hard-won personalized models are not well exploited and propose "inherited private model" to store the personalization experience. Our proposed algorithm named MAP could simultaneously achieve the aggregation and personalization goals in FL. Abundant experimental studies verify the superiorities of our algorithm.