Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CollaFuse: Navigating Limited Resources and Privacy in Collaborative Generative AI

Domenique Zipperling, Simeon Allmendinger, Lukas Struppek, Niklas Kühl

TL;DR

CollaFuse tackles the data, privacy, and compute bottlenecks of diffusion-based GenAI in distributed settings by introducing a split-learning–inspired framework that partitions the heavy denoising computation on a shared server while keeping data locally. It uses a cut-ratio $c$ to control the denoising split at $t_c=(1-c)T$, enabling a backbone server and client-specific heads and enabling both training and inference collaboration. The paper provides MRI-based experiments showing improved image fidelity with collaboration, reduced local energy use, and manageable information disclosure, highlighting a trade-off curve with $c$. The work offers a practical blueprint for edge-oriented, privacy-preserving collaborative GenAI and outlines future directions toward color imagery, text-guided generation, and differential privacy safeguards.

Abstract

In the landscape of generative artificial intelligence, diffusion-based models present challenges for socio-technical systems in data requirements and privacy. Traditional approaches like federated learning distribute the learning process but strain individual clients, especially with constrained resources (e.g., edge devices). In response to these challenges, we introduce CollaFuse, a novel framework inspired by split learning. Tailored for efficient and collaborative use of denoising diffusion probabilistic models, CollaFuse enables shared server training and inference, alleviating client computational burdens. This is achieved by retaining data and computationally inexpensive GPU processes locally at each client while outsourcing the computationally expensive processes to the shared server. Demonstrated in a healthcare context, CollaFuse enhances privacy by highly reducing the need for sensitive information sharing. These capabilities hold the potential to impact various application areas, such as the design of edge computing solutions, healthcare research, or autonomous driving. In essence, our work advances distributed machine learning, shaping the future of collaborative GenAI networks.

CollaFuse: Navigating Limited Resources and Privacy in Collaborative Generative AI

TL;DR

CollaFuse tackles the data, privacy, and compute bottlenecks of diffusion-based GenAI in distributed settings by introducing a split-learning–inspired framework that partitions the heavy denoising computation on a shared server while keeping data locally. It uses a cut-ratio to control the denoising split at , enabling a backbone server and client-specific heads and enabling both training and inference collaboration. The paper provides MRI-based experiments showing improved image fidelity with collaboration, reduced local energy use, and manageable information disclosure, highlighting a trade-off curve with . The work offers a practical blueprint for edge-oriented, privacy-preserving collaborative GenAI and outlines future directions toward color imagery, text-guided generation, and differential privacy safeguards.

Abstract

In the landscape of generative artificial intelligence, diffusion-based models present challenges for socio-technical systems in data requirements and privacy. Traditional approaches like federated learning distribute the learning process but strain individual clients, especially with constrained resources (e.g., edge devices). In response to these challenges, we introduce CollaFuse, a novel framework inspired by split learning. Tailored for efficient and collaborative use of denoising diffusion probabilistic models, CollaFuse enables shared server training and inference, alleviating client computational burdens. This is achieved by retaining data and computationally inexpensive GPU processes locally at each client while outsourcing the computationally expensive processes to the shared server. Demonstrated in a healthcare context, CollaFuse enhances privacy by highly reducing the need for sensitive information sharing. These capabilities hold the potential to impact various application areas, such as the design of edge computing solutions, healthcare research, or autonomous driving. In essence, our work advances distributed machine learning, shaping the future of collaborative GenAI networks.
Paper Structure (6 sections, 3 figures)

This paper contains 6 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Framework
  4. Experimental Evaluation
  5. Results
  6. Conclusion and Outlook

Figures (3)

  • Figure 1: Illustration of the denoising process in DDPMs: Exemplary images generated at denoising step $t$ for various cut-ratios $c$. The distinguishing features of the generated images remain effectively concealed behind noise during the majority of denoising steps.
  • Figure 2: The training procedure of CollaFuse comprises six steps from client perspective (a) and system (b): Server triggers diffusion process of clients (1), clients apply diffusion (2), clients send diffused images and noise to server (3), server denoises the images until $t_c$ (4), server sends the partially denoised images to client (5), and clients locally finish denoising process (6).
  • Figure 3: Preliminary results: We calculate the KID score to assess generative performance and out-of-sample robustness (upper-left, lower-left). Information disclosure is evaluated by comparing original client data to server-generated images, using KID for distribution (upper-right) and MSE for pixel-level comparison. Despite conducting up to 80% of the computationally intensive denoising steps on the server, a significant amount of information in the images remains concealed.