FedRD: Reducing Divergences for Generalized Federated Learning via Heterogeneity-aware Parameter Guidance

TL;DR

FedRD tackles generalized federated learning under heterogeneous data by identifying optimization divergence and performance divergence as key obstacles. It introduces a two-pronged approach: a debiased local classifier to mitigate class-imbalance effects and a heterogeneity-aware global aggregation that uses domain discrepancy cues and a GAP-based compensation to guide updates. The method encodes domain knowledge in the feature extractor and uses adaptive reweighting and a probabilistic aggregation scheme to improve generalization to unseen clients. Experiments on four public multi-domain datasets show substantial improvements over strong baselines, validating FedRD's effectiveness in reducing divergences and enhancing cross-domain generalization. The work offers a foundation for further exploration of domain-aware aggregation under privacy constraints.

Abstract

Heterogeneous federated learning (HFL) aims to ensure effective and privacy-preserving collaboration among different entities. As newly joined clients require significant adjustments and additional training to align with the existing system, the problem of generalizing federated learning models to unseen clients under heterogeneous data has become progressively crucial. Consequently, we highlight two unsolved challenging issues in federated domain generalization: Optimization Divergence and Performance Divergence. To tackle the above challenges, we propose FedRD, a novel heterogeneity-aware federated learning algorithm that collaboratively utilizes parameter-guided global generalization aggregation and local debiased classification to reduce divergences, aiming to obtain an optimal global model for participating and unseen clients. Extensive experiments on public multi-domain datasets demonstrate that our approach exhibits a substantial performance advantage over competing baselines in addressing this specific problem.

Figures (6)

• Figure 1: Overview of FedRD framework. Let $C$ represents the number of classes within the client datasets, the local model, denoted as $f_i(w)=f_{\phi}^{(i)}\circ f_{\psi}^{(i)}$, is composed of a feature extractor $f_{\phi}^{(i)}: \mathbb{R}^X \rightarrow \mathbb{R}^D$ and a debiased classifier $f_{\psi}^{(i)}: \mathbb{R}^D \rightarrow \mathbb{R}^C$. On the server side, the global generalization aggregation strategy updates the global model $f_g(w)$.
• Figure 2: Visualization of images activating each filter in different layers. Best viewed electronically, zoomed in.
• Figure 3: Training performance under different communication rounds.
• Figure 4: Training performance and feature extraction analysis.
• Figure 5: Training Performance under different heterogeneity.