Federated Source-free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Data

TL;DR

This work tackles Federated source-Free Domain Adaptation (FFREEDA) for classification, where a server holds a source-trained model and clients possess unlabeled data from diverse target domains. It introduces FedWCA, a three-phase method: (1) private, parameter-free clustering of clients using only first-layer parameters via FINCH to form domain-based clusters, (2) Weighted Cluster Aggregation that blends cluster models with client-specific weights computed from prototypes and Soft Neighborhood Density, and (3) local adaptation with a two-phase pseudo-labeling strategy incorporating prototypes and mixup to leverage unlabeled data. Empirically, FedWCA outperforms baselines including LADD and FedPCL+PL on Digit-Five, PACS, and Office-Home, with ablations confirming the value of cluster weighting, clustering layers, and the pseudo-labeling design. The method advances practical FFREEDA by reducing privacy risks, minimizing hyperparameters, and enabling effective cross-domain knowledge transfer for personalized classification in non-i.i.d. federated settings.

Abstract

Federated learning (FL) commonly assumes that the server or some clients have labeled data, which is often impractical due to annotation costs and privacy concerns. Addressing this problem, we focus on a source-free domain adaptation task, where (1) the server holds a pre-trained model on labeled source domain data, (2) clients possess only unlabeled data from various target domains, and (3) the server and clients cannot access the source data in the adaptation phase. This task is known as Federated source-Free Domain Adaptation (FFREEDA). Specifically, we focus on classification tasks, while the previous work solely studies semantic segmentation. Our contribution is the novel Federated learning with Weighted Cluster Aggregation (FedWCA) method, designed to mitigate both domain shifts and privacy concerns with only unlabeled data. FedWCA comprises three phases: private and parameter-free clustering of clients to obtain domain-specific global models on the server, weighted aggregation of the global models for the clustered clients, and local domain adaptation with pseudo-labeling. Experimental results show that FedWCA surpasses several existing methods and baselines in FFREEDA, establishing its effectiveness and practicality.

Figures (6)

• Figure 1: Overview of our problem setting: FFREEDA shenaj_wacv2023_ladd. A pre-trained model on a labeled source domain dataset exists on the server, which becomes inaccessible following pre-training (source-free). Clients possess unlabeled datasets from specific target domains, which may or may not be identical. We aim to train personalized models tailored to each client's domain.
• Figure 2: Weighted cluster aggregation (WCA). Our WCA consists of server-side (left) and client-side (right) operations. The client-side operations are represented by client $1$ in cluster $1$. Firstly, the server forms cluster models by averaging the local models within each cluster and develops soft cluster models by integrating models from other clusters. Next, the client merges these soft cluster models utilizing cluster weights determined by how each feature extractor aligns each sample with the respective classifier vector. Additionally, the client combines the merged model with the corresponding cluster model using Soft Neighborhood Density (SND) weights snd.
• Figure 3: Pseudo-labeling strategy. The operation is signified by client $1$ in cluster $1$. Our strategy involves two phases: prototype-based pseudo-labeling with initial and cluster models, and mixup to generate samples with more reliable pseudo-labels. The first phase selects pseudo-labels closer to prototypes, while the second mixes samples with mismatched and matched pseudo-labels.
• Figure 4: Comparison of average accuracy over clients within each target domain. The source domain is MNIST for Digit-Five (top) and Art Painting for PACS (bottom). The target domains are the others. FedWCA considerably enhances the accuracy of clients in specific target domains such as SVHN for Digit-Five and Sketch for PACS, while maintaining the performance of other domains.
• Figure 5: Visualization of cluster weights for FedWCA-L and FedWCA when the communication round $r$ is $2$ and $50$. Art Painting in PACS dataset is used for the source dataset. Clients $C, C', C"$ belong to Cartoon, $P, P', P"$ to Photo, and $S, S', S"$ to Sketch. See the first row of \ref{['table:clustering_result']} for the cluster IDs (denoted by $\bigcirc$, $\triangle$, and $\square$) assigned to each client. The number represents the weight for the corresponding cluster.