Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Model

Mingzhao Yang, Shangchao Su, Bin Li, Xiangyang Xue

TL;DR

This work tackles practical one-shot semi-Federated Learning where clients hold unlabeled data and the server has labeled data, a setting hindered by heavy communication, data heterogeneity, and device constraints. It introduces FedDISC, a diffusion-model–guided co-training framework that generates server-side synthetic data conditioned by client distributions without any client training. The method uses four steps—Prototype Extraction, Pseudo Labeling, Feature Processing, and Image Generation with domain-aware guidance and privacy-preserving noise—to create diverse, high-quality data that enable a global model to approach or exceed centralized supervision in a single communication round. Experiments on DomainNet, OpenImage, and NICO++ show FedDISC outperforms state-of-the-art semi-FL methods and achieves competitive or superior results compared to centralized training, while visual analyses indicate strong distribution alignment and minimal privacy leakage.

Abstract

Recently, semi-supervised federated learning (semi-FL) has been proposed to handle the commonly seen real-world scenarios with labeled data on the server and unlabeled data on the clients. However, existing methods face several challenges such as communication costs, data heterogeneity, and training pressure on client devices. To address these challenges, we introduce the powerful diffusion models (DM) into semi-FL and propose FedDISC, a Federated Diffusion-Inspired Semi-supervised Co-training method. Specifically, we first extract prototypes of the labeled server data and use these prototypes to predict pseudo-labels of the client data. For each category, we compute the cluster centroids and domain-specific representations to signify the semantic and stylistic information of their distributions. After adding noise, these representations are sent back to the server, which uses the pre-trained DM to generate synthetic datasets complying with the client distributions and train a global model on it. With the assistance of vast knowledge within DM, the synthetic datasets have comparable quality and diversity to the client images, subsequently enabling the training of global models that achieve performance equivalent to or even surpassing the ceiling of supervised centralized training. FedDISC works within one communication round, does not require any local training, and involves very minimal information uploading, greatly enhancing its practicality. Extensive experiments on three large-scale datasets demonstrate that FedDISC effectively addresses the semi-FL problem on non-IID clients and outperforms the compared SOTA methods. Sufficient visualization experiments also illustrate that the synthetic dataset generated by FedDISC exhibits comparable diversity and quality to the original client dataset, with a neglectable possibility of leaking privacy-sensitive information of the clients.

Exploring One-shot Semi-supervised Federated Learning with A Pre-trained Diffusion Model

TL;DR

This work tackles practical one-shot semi-Federated Learning where clients hold unlabeled data and the server has labeled data, a setting hindered by heavy communication, data heterogeneity, and device constraints. It introduces FedDISC, a diffusion-model–guided co-training framework that generates server-side synthetic data conditioned by client distributions without any client training. The method uses four steps—Prototype Extraction, Pseudo Labeling, Feature Processing, and Image Generation with domain-aware guidance and privacy-preserving noise—to create diverse, high-quality data that enable a global model to approach or exceed centralized supervision in a single communication round. Experiments on DomainNet, OpenImage, and NICO++ show FedDISC outperforms state-of-the-art semi-FL methods and achieves competitive or superior results compared to centralized training, while visual analyses indicate strong distribution alignment and minimal privacy leakage.

Abstract

Recently, semi-supervised federated learning (semi-FL) has been proposed to handle the commonly seen real-world scenarios with labeled data on the server and unlabeled data on the clients. However, existing methods face several challenges such as communication costs, data heterogeneity, and training pressure on client devices. To address these challenges, we introduce the powerful diffusion models (DM) into semi-FL and propose FedDISC, a Federated Diffusion-Inspired Semi-supervised Co-training method. Specifically, we first extract prototypes of the labeled server data and use these prototypes to predict pseudo-labels of the client data. For each category, we compute the cluster centroids and domain-specific representations to signify the semantic and stylistic information of their distributions. After adding noise, these representations are sent back to the server, which uses the pre-trained DM to generate synthetic datasets complying with the client distributions and train a global model on it. With the assistance of vast knowledge within DM, the synthetic datasets have comparable quality and diversity to the client images, subsequently enabling the training of global models that achieve performance equivalent to or even surpassing the ceiling of supervised centralized training. FedDISC works within one communication round, does not require any local training, and involves very minimal information uploading, greatly enhancing its practicality. Extensive experiments on three large-scale datasets demonstrate that FedDISC effectively addresses the semi-FL problem on non-IID clients and outperforms the compared SOTA methods. Sufficient visualization experiments also illustrate that the synthetic dataset generated by FedDISC exhibits comparable diversity and quality to the original client dataset, with a neglectable possibility of leaking privacy-sensitive information of the clients.
Paper Structure (15 sections, 10 equations, 5 figures, 5 tables)

This paper contains 15 sections, 10 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Federated Learning
  4. Foundation Models
  5. FL with Foundation Models
  6. Method
  7. Preliminaries
  8. FedDISC
  9. Experiments
  10. Experimental Setup
  11. Main Results
  12. Ablation Experiments
  13. Discussions
  14. Conclusion
  15. Acknowledgements

Figures (5)

  • Figure 1: The framework of FedDISC. The overall method consists of four steps: Prototype Extraction, Pseudo Labeling, Feature Processing, and Image Generation.
  • Figure 2: Generated images comply with different distributions on different datasets.
  • Figure 3: Comparison between generating using clustering centroids and the randomly selected client representations. With the provision of clustering centroids, the introduction of more representative semantic information leads to a significant improvement in the stability of the generated outputs.
  • Figure 4: The inclusion of domain-specific representations and their impact on the generated results. We can effectively alter the style of the generated images by controlling the added domain-specific representations, thereby enhancing the diversity of generated samples.
  • Figure 5: The comparison between the raw client images and their generated images. It can be observed that the generated images do not leak any sensitive privacy present in the original images, such as faces, text, etc. The generated images exhibit only a stylistic resemblance to the original images. Restoring original images starting from high-dimensional features without any training is nearly impossible.