FedPall: Prototype-based Adversarial and Collaborative Learning for Federated Learning with Feature Drift

TL;DR

This work tackles feature drift in federated learning by introducing FedPall, a prototype-based adversarial and collaborative framework that unifies heterogeneous feature spaces while preserving privacy. Key ideas include generating global class prototypes, adversarially aligning client features with a server-side amplifier, and reinforcing class information via InfoNCE with global prototypes. A novel prototype-mixing mechanism creates global-perspective features that train a server-side global classifier, which is then decentralized back to clients for improved personalization. Empirical results on three feature-drift benchmarks show state-of-the-art performance, with ablations confirming the necessity of combining adversarial alignment, prototype-based collaboration, and a global classifier for robust cross-client performance.

Abstract

Federated learning (FL) enables collaborative training of a global model in the centralized server with data from multiple parties while preserving privacy. However, data heterogeneity can significantly degrade the performance of the global model when each party uses datasets from different sources to train a local model, thereby affecting personalized local models. Among various cases of data heterogeneity, feature drift, feature space difference among parties, is prevalent in real-life data but remains largely unexplored. Feature drift can distract feature extraction learning in clients and thus lead to poor feature extraction and classification performance. To tackle the problem of feature drift in FL, we propose FedPall, an FL framework that utilizes prototype-based adversarial learning to unify feature spaces and collaborative learning to reinforce class information within the features. Moreover, FedPall leverages mixed features generated from global prototypes and local features to enhance the global classifier with classification-relevant information from a global perspective. Evaluation results on three representative feature-drifted datasets demonstrate FedPall's consistently superior performance in classification with feature-drifted data in the FL scenario.

Figures (6)

• Figure 1: We show a schematic diagram of feature drift and use different techniques to drive feature distribution to update in different directions.
• Figure 2: The FedPall framework and detailed phases
• Figure 3: We evaluate the top-1 accuracy averaged over all clients using different loss function combinations on different datasets.
• Figure 4: We plotted the feature distribution of different categories under different clients, corresponding to the four loss combination strategies of \ref{['fig:ablation-office']}
• Figure 5: Acc. of different $\mu$ and $\delta$ under Office-10 dataset