FedSCAl: Leveraging Server and Client Alignment for Unsupervised Federated Source-Free Domain Adaptation

TL;DR

FFreeDA addresses unsupervised domain adaptation across unlabeled, heterogeneous clients with a server-side pre-trained model. FedSCAl introduces Server-Client Alignment (SCAl) to regularize client updates by aligning predictions with the server model, mitigating client-drift and improving pseudo-label quality, complemented by adaptive thresholding. The method demonstrates consistent gains over state-of-the-art FL baselines on Office-Home, DomainNet, and Office-31 across various server initializations. Overall, FedSCAl offers a robust, clustering-free approach for cross-domain federated classification under limited server access to labeled data, with practical improvements in real-world heterogeneous deployments.

Abstract

We address the Federated source-Free Domain Adaptation (FFreeDA) problem, with clients holding unlabeled data with significant inter-client domain gaps. The FFreeDA setup constrains the FL frameworks to employ only a pre-trained server model as the setup restricts access to the source dataset during the training rounds. Often, this source domain dataset has a distinct distribution to the clients' domains. To address the challenges posed by the FFreeDA setup, adaptation of the Source-Free Domain Adaptation (SFDA) methods to FL struggles with client-drift in real-world scenarios due to extreme data heterogeneity caused by the aforementioned domain gaps, resulting in unreliable pseudo-labels. In this paper, we introduce FedSCAl, an FL framework leveraging our proposed Server-Client Alignment (SCAl) mechanism to regularize client updates by aligning the clients' and server model's predictions. We observe an improvement in the clients' pseudo-labeling accuracy post alignment, as the SCAl mechanism helps to mitigate the client-drift. Further, we present extensive experiments on benchmark vision datasets showcasing how FedSCAl consistently outperforms state-of-the-art FL methods in the FFreeDA setup for classification tasks.

Figures (6)

• Figure 1: (a) FFreeDA Setup:Multiple clients (labeled as $\text{C}_1$, $\text{C}_2$, $\text{C}_3$, and $\text{C}_4$) hold unlabeled data from same or different domains, while the central server uses a pre-trained model trained on a labeled source dataset, which differs from clients' data distributions and is unavailable during training. (b) Performance of different methods on Office-Home dataset.
• Figure 1: (a) t-SNE plot of representations learned by FedLoA, (b) t-SNE plot for representations learned after adding our proposed loss (FedSCAl). It can be seen that adding our SCAl loss leads to better clustering of the learned representations.
• Figure 2: Overview of the proposed FedSCAl framework with Server and Client Alignment (SCAl) implementation. The server communicates the global model $\mathbf{w}^r$ to the clients, the clients then train their local models ($\mathbf{w}_k^r$) using a weak augmentation $A_w$ giving us the augmented image $A_w(x_k^i)$, and a strong augmentation $A_s$ from which we get the augmented image $A_s(x_k^i)$. The clients compute the client alignment loss: $L_k^{l,i}$ and the server alignment loss: $L_k^{g,i}$ to perform the local training using the Eq. \ref{['client_loss']}, and then communicate the updated local models $w_k^r$ to the server in the subsequent round for aggregation.
• Figure 2: Ablation of SCAl Regularization Weights: Average accuracy across clients while varying $\lambda_l$ and $\lambda_g$.
• Figure 3: Pseudo-label Accuracy difference and visualization of Entropy Density of Clients Predictions.