Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiffusionX: Efficient Edge-Cloud Collaborative Image Generation with Multi-Round Prompt Evolution

Yi Wei, Shunpu Tang, Liang Zhao, Qiangian Yang

TL;DR

DiffusionX tackles the latency and resource demands of diffusion-based text-to-image generation by distributing work between a lightweight edge model for rapid previews and a high-capacity cloud model for refinement. It introduces a semantic-aware strength predictor on the edge and a multimodal, skip-step denoising predictor on the cloud to support efficient, multi-round prompt evolution. Experiments show a 15.8% reduction in average generation time versus cloud-only SD v1.5 while maintaining competitive image quality, and performance close to Tiny-SD with significantly improved fidelity. Overall, DiffusionX provides a scalable, low-latency edge–cloud framework that preserves image quality with minimal overhead for interactive T2I applications.

Abstract

Recent advances in diffusion models have driven remarkable progress in image generation. However, the generation process remains computationally intensive, and users often need to iteratively refine prompts to achieve the desired results, further increasing latency and placing a heavy burden on cloud resources. To address this challenge, we propose DiffusionX, a cloud-edge collaborative framework for efficient multi-round, prompt-based generation. In this system, a lightweight on-device diffusion model interacts with users by rapidly producing preview images, while a high-capacity cloud model performs final refinements after the prompt is finalized. We further introduce a noise level predictor that dynamically balances the computation load, optimizing the trade-off between latency and cloud workload. Experiments show that DiffusionX reduces average generation time by 15.8% compared with Stable Diffusion v1.5, while maintaining comparable image quality. Moreover, it is only 0.9% slower than Tiny-SD with significantly improved image quality, thereby demonstrating efficiency and scalability with minimal overhead.

DiffusionX: Efficient Edge-Cloud Collaborative Image Generation with Multi-Round Prompt Evolution

TL;DR

DiffusionX tackles the latency and resource demands of diffusion-based text-to-image generation by distributing work between a lightweight edge model for rapid previews and a high-capacity cloud model for refinement. It introduces a semantic-aware strength predictor on the edge and a multimodal, skip-step denoising predictor on the cloud to support efficient, multi-round prompt evolution. Experiments show a 15.8% reduction in average generation time versus cloud-only SD v1.5 while maintaining competitive image quality, and performance close to Tiny-SD with significantly improved fidelity. Overall, DiffusionX provides a scalable, low-latency edge–cloud framework that preserves image quality with minimal overhead for interactive T2I applications.

Abstract

Recent advances in diffusion models have driven remarkable progress in image generation. However, the generation process remains computationally intensive, and users often need to iteratively refine prompts to achieve the desired results, further increasing latency and placing a heavy burden on cloud resources. To address this challenge, we propose DiffusionX, a cloud-edge collaborative framework for efficient multi-round, prompt-based generation. In this system, a lightweight on-device diffusion model interacts with users by rapidly producing preview images, while a high-capacity cloud model performs final refinements after the prompt is finalized. We further introduce a noise level predictor that dynamically balances the computation load, optimizing the trade-off between latency and cloud workload. Experiments show that DiffusionX reduces average generation time by 15.8% compared with Stable Diffusion v1.5, while maintaining comparable image quality. Moreover, it is only 0.9% slower than Tiny-SD with significantly improved image quality, thereby demonstrating efficiency and scalability with minimal overhead.
Paper Structure (11 sections, 4 equations, 4 figures, 4 tables)

This paper contains 11 sections, 4 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Proposed System
  3. Fast Preview with Noise Level Predictor
  4. High-Fidelity Noise Level Predictor with Skip-Step Acceleration
  5. EXPERIMENTS
  6. Experimental Setup
  7. Image Generation Quality
  8. Image Generation Efficiency
  9. Ablation Studies
  10. CONCLUSIONS
  11. ACKNOWLEDGEMENT

Figures (4)

  • Figure 1: Illustration of the proposed DiffusionX framework, where the edge model provides fast previews for user interaction and the cloud model refines them into high-fidelity results.
  • Figure 2: The structure of img2img generation on the edge with a noise level predictor.
  • Figure 3: The structure of img2img generation on the cloud with a noise level predictor for refinement.
  • Figure 4: Visual examples of images generated by SD v1.5 and the proposed DiffusionX on the COCO2017-Interactive-Prompts-400 Dataset.