Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One-Shot Federated Learning with Classifier-Free Diffusion Models

Obaidullah Zaland, Shutong Jin, Florian T. Pokorny, Monowar Bhuyan

TL;DR

OSCAR tackles the high communication cost of federated learning by enabling one shot global model training through server side data generation guided by classifier free diffusion and foundation models. It uses BLIP to describe client data and CLIP to encode these descriptions, conditioning a diffusion model without auxiliary classifiers, and synthesizes a global dataset for centralized training of a ResNet-18. The method achieves superior performance on four benchmarks while reducing client upload by at least 99%, and it remains effective under non IID data distributions. This approach enhances scalability and efficiency in FL by shifting data synthesis to the server and leveraging pre trained encoding models.

Abstract

Federated learning (FL) enables collaborative learning without data centralization but introduces significant communication costs due to multiple communication rounds between clients and the server. One-shot federated learning (OSFL) addresses this by forming a global model with a single communication round, often relying on the server's model distillation or auxiliary dataset generation - often through pre-trained diffusion models (DMs). Existing DM-assisted OSFL methods, however, typically employ classifier-guided DMs, which require training auxiliary classifier models at each client, introducing additional computation overhead. This work introduces OSCAR (One-Shot Federated Learning with Classifier-Free Diffusion Models), a novel OSFL approach that eliminates the need for auxiliary models. OSCAR uses foundation models to devise category-specific data representations at each client, seamlessly integrated into a classifier-free diffusion model pipeline for server-side data generation. OSCAR is a simple yet cost-effective OSFL approach that outperforms the state-of-the-art on four benchmarking datasets while reducing the communication load by at least 99%.

One-Shot Federated Learning with Classifier-Free Diffusion Models

TL;DR

OSCAR tackles the high communication cost of federated learning by enabling one shot global model training through server side data generation guided by classifier free diffusion and foundation models. It uses BLIP to describe client data and CLIP to encode these descriptions, conditioning a diffusion model without auxiliary classifiers, and synthesizes a global dataset for centralized training of a ResNet-18. The method achieves superior performance on four benchmarks while reducing client upload by at least 99%, and it remains effective under non IID data distributions. This approach enhances scalability and efficiency in FL by shifting data synthesis to the server and leveraging pre trained encoding models.

Abstract

Federated learning (FL) enables collaborative learning without data centralization but introduces significant communication costs due to multiple communication rounds between clients and the server. One-shot federated learning (OSFL) addresses this by forming a global model with a single communication round, often relying on the server's model distillation or auxiliary dataset generation - often through pre-trained diffusion models (DMs). Existing DM-assisted OSFL methods, however, typically employ classifier-guided DMs, which require training auxiliary classifier models at each client, introducing additional computation overhead. This work introduces OSCAR (One-Shot Federated Learning with Classifier-Free Diffusion Models), a novel OSFL approach that eliminates the need for auxiliary models. OSCAR uses foundation models to devise category-specific data representations at each client, seamlessly integrated into a classifier-free diffusion model pipeline for server-side data generation. OSCAR is a simple yet cost-effective OSFL approach that outperforms the state-of-the-art on four benchmarking datasets while reducing the communication load by at least 99%.

Paper Structure

This paper contains 20 sections, 9 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Prior Work
  3. One-Shot Federated Learning
  4. Federated Learning with Foundation Models
  5. Preliminaries
  6. Diffusion Models
  7. OSCAR: One Shot Federated Learning with Classifier-Free Diffusion Models
  8. Description Generation and Text Encoding
  9. Client Representation and Server Data Synthesis
  10. Model Training
  11. Experimental Setup
  12. Datasets
  13. Data Division
  14. Baselines
  15. Results and Analysis
  16. ...and 5 more sections

Figures (2)

  • Figure 1: The number of uploaded parameters by each client and accuracy for various algorithms on OpenImage dataset and ResNet-18.
  • Figure 2: An illustration of the proposed OSCAR pipeline, where BLIP li2022blip, CLIP radford2021learning Text Encoder, and Stable Diffusion rombach2022high are all used with frozen weights and in a zero-shot manner.