Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training
Milad Soltany, Farhad Pourpanah, Mahdiyar Molahasani, Michael Greenspan, Ali Etemad
TL;DR
The paper addresses data heterogeneity in federated domain generalization, where unseen domains challenge a globally learned model. FedSB combines client-level label smoothing to curb overconfident, domain-specific predictions and a fixed-budget training scheme to balance contributions across $K$ clients, updating the global parameters $\Theta^{t+1}$. It achieves state-of-the-art performance on 3 of 4 domain-generalization benchmarks (PACS, OfficeHome, TerraIncognita, VLCS) across multiple backbones, with ablations validating the contribution of each component and sensitivity analyses showing robustness to $\epsilon$ and $S$. The work advances privacy-preserving multi-domain learning by enabling more robust, generalizable models and provides publicly available code.
Abstract
In this paper, we propose a novel approach, Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training (FedSB), to address the challenges of data heterogeneity within a federated learning framework. FedSB utilizes label smoothing at the client level to prevent overfitting to domain-specific features, thereby enhancing generalization capabilities across diverse domains when aggregating local models into a global model. Additionally, FedSB incorporates a decentralized budgeting mechanism which balances training among clients, which is shown to improve the performance of the aggregated global model. Extensive experiments on four commonly used multi-domain datasets, PACS, VLCS, OfficeHome, and TerraInc, demonstrate that FedSB outperforms competing methods, achieving state-of-the-art results on three out of four datasets, indicating the effectiveness of FedSB in addressing data heterogeneity.