Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Models

Mingzhao Yang, Shangchao Su, Bin Li, Xiangyang Xue

TL;DR

FedDEO tackles one-shot federated learning by replacing traditional data transfer mediums with learnable per-client distribution descriptions and conditioning a fixed pre-trained diffusion model to generate synthetic data that match client distributions. Each client trains per-category description vectors using the DM, and the server synthesizes labeled data guided by these descriptions to train a global model. A KL-divergence-based bound connects the synthetic data distribution to client data, supporting the method's theoretical validity. Empirically, FedDEO outperforms several baselines and even surpasses centralized training ceilings on realistic datasets, while reducing upload communication and preserving privacy. Overall, the approach demonstrates how diffusion models can unlock privacy-preserving, communication-efficient OSFL with high practical impact.

Abstract

In recent years, the attention towards One-Shot Federated Learning (OSFL) has been driven by its capacity to minimize communication. With the development of the diffusion model (DM), several methods employ the DM for OSFL, utilizing model parameters, image features, or textual prompts as mediums to transfer the local client knowledge to the server. However, these mediums often require public datasets or the uniform feature extractor, significantly limiting their practicality. In this paper, we propose FedDEO, a Description-Enhanced One-Shot Federated Learning Method with DMs, offering a novel exploration of utilizing the DM in OSFL. The core idea of our method involves training local descriptions on the clients, serving as the medium to transfer the knowledge of the distributed clients to the server. Firstly, we train local descriptions on the client data to capture the characteristics of client distributions, which are then uploaded to the server. On the server, the descriptions are used as conditions to guide the DM in generating synthetic datasets that comply with the distributions of various clients, enabling the training of the aggregated model. Theoretical analyses and sufficient quantitation and visualization experiments on three large-scale real-world datasets demonstrate that through the training of local descriptions, the server is capable of generating synthetic datasets with high quality and diversity. Consequently, with advantages in communication and privacy protection, the aggregated model outperforms compared FL or diffusion-based OSFL methods and, on some clients, outperforms the performance ceiling of centralized training.

FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Models

TL;DR

FedDEO tackles one-shot federated learning by replacing traditional data transfer mediums with learnable per-client distribution descriptions and conditioning a fixed pre-trained diffusion model to generate synthetic data that match client distributions. Each client trains per-category description vectors using the DM, and the server synthesizes labeled data guided by these descriptions to train a global model. A KL-divergence-based bound connects the synthetic data distribution to client data, supporting the method's theoretical validity. Empirically, FedDEO outperforms several baselines and even surpasses centralized training ceilings on realistic datasets, while reducing upload communication and preserving privacy. Overall, the approach demonstrates how diffusion models can unlock privacy-preserving, communication-efficient OSFL with high practical impact.

Abstract

In recent years, the attention towards One-Shot Federated Learning (OSFL) has been driven by its capacity to minimize communication. With the development of the diffusion model (DM), several methods employ the DM for OSFL, utilizing model parameters, image features, or textual prompts as mediums to transfer the local client knowledge to the server. However, these mediums often require public datasets or the uniform feature extractor, significantly limiting their practicality. In this paper, we propose FedDEO, a Description-Enhanced One-Shot Federated Learning Method with DMs, offering a novel exploration of utilizing the DM in OSFL. The core idea of our method involves training local descriptions on the clients, serving as the medium to transfer the knowledge of the distributed clients to the server. Firstly, we train local descriptions on the client data to capture the characteristics of client distributions, which are then uploaded to the server. On the server, the descriptions are used as conditions to guide the DM in generating synthetic datasets that comply with the distributions of various clients, enabling the training of the aggregated model. Theoretical analyses and sufficient quantitation and visualization experiments on three large-scale real-world datasets demonstrate that through the training of local descriptions, the server is capable of generating synthetic datasets with high quality and diversity. Consequently, with advantages in communication and privacy protection, the aggregated model outperforms compared FL or diffusion-based OSFL methods and, on some clients, outperforms the performance ceiling of centralized training.
Paper Structure (15 sections, 8 equations, 4 figures, 5 tables)

This paper contains 15 sections, 8 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Diffusion Model
  4. One-Shot Federated Learning
  5. Method
  6. Preliminaries
  7. Client Description Training
  8. Server Image Generation
  9. Theoretical Analysis
  10. Experiments
  11. Experimental Settings
  12. Main Results
  13. Ablation Experiments
  14. Limitations and Discussions
  15. Conclusions

Figures (4)

  • Figure 1: The overall framework of FedDEO, including two main parts: Client Description Training and Server Image Generation. Firstly, each client trains local descriptions based on the client data and the pre-trained diffusion model, then uploads them to the server. Guided by these descriptions, the server utilizes the diffusion model to generate the synthetic dataset that complies with the various client distributions and trains the aggregated model.
  • Figure 2: The visualization of generated samples on DomainNet and OpenImage.
  • Figure 3: The visualization of generated samples on NICO++.
  • Figure 4: The visualization of privacy-sensitive information-related categories.