Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text-Guided Diffusion Model-based Generative Communication for Wireless Image Transmission

Shengkang Chen, Tong Wu, Zhiyong Chen, Feng Yang, Meixia Tao, Wenjun Zhang

TL;DR

This paper tackles ultra-low bitrate wireless image transmission by marrying latent-domain JSCC with diffusion-based reconstruction guided by semantic prompts. It encodes images in a latent space via the Stable Diffusion VAE and transmits a JSCC-encoded latent alongside a semantic prompt o generated by an multimodal LLM (e.g., GPT-4o), with the prompt protected by arithmetic coding and LDPC. At the receiver, a warm-start diffusion process refines the JSCC latent z_c using both the structural cue from z_c and the semantic cue o, enabling high-quality reconstructions in as few as 5 denoising steps without retraining the diffusion model. Experiments on DIV2K and Kodak show superior perceptual and distributional fidelity over state-of-the-art JSCC baselines across AWGN and Rayleigh channels, with strong scalability to higher resolutions thanks to the latent-domain design.

Abstract

Reliable image transmission over wireless channels is particularly challenging at extremely low transmission rates, where conventional compression and channel coding schemes fail to preserve adequate visual quality. To address this issue, we propose a generative communication framework based on diffusion models, which integrates joint source channel coding (JSCC) with semantic-guided reconstruction leveraging a pre-trained generative model. Unlike conventional architectures that aim to recover exact pixel values of the original image, the proposed method focuses on preserving and reconstructing semantically meaningful visual content under severely constrained rates, ensuring perceptual plausibility and faithfulness to the scene intent. Specifically, the transmitter encodes the source image via JSCC and jointly transmits it with a textual prompt over the wireless channel. At the receiver, the corrupted low-rate representation is fused with the prompt and reconstructed through a Stable Diffusion model with ControlNet, enabling high-quality visual recovery. Leveraging both generative priors and semantic guidance, the proposed framework produces perceptually convincing images even under extreme bandwidth limitations. Experimental results demonstrate that the proposed method consistently outperforms conventional coding-based schemes and deep learning baselines, achieving superior perceptual quality and robustness across various channel conditions.

Text-Guided Diffusion Model-based Generative Communication for Wireless Image Transmission

TL;DR

This paper tackles ultra-low bitrate wireless image transmission by marrying latent-domain JSCC with diffusion-based reconstruction guided by semantic prompts. It encodes images in a latent space via the Stable Diffusion VAE and transmits a JSCC-encoded latent alongside a semantic prompt o generated by an multimodal LLM (e.g., GPT-4o), with the prompt protected by arithmetic coding and LDPC. At the receiver, a warm-start diffusion process refines the JSCC latent z_c using both the structural cue from z_c and the semantic cue o, enabling high-quality reconstructions in as few as 5 denoising steps without retraining the diffusion model. Experiments on DIV2K and Kodak show superior perceptual and distributional fidelity over state-of-the-art JSCC baselines across AWGN and Rayleigh channels, with strong scalability to higher resolutions thanks to the latent-domain design.

Abstract

Reliable image transmission over wireless channels is particularly challenging at extremely low transmission rates, where conventional compression and channel coding schemes fail to preserve adequate visual quality. To address this issue, we propose a generative communication framework based on diffusion models, which integrates joint source channel coding (JSCC) with semantic-guided reconstruction leveraging a pre-trained generative model. Unlike conventional architectures that aim to recover exact pixel values of the original image, the proposed method focuses on preserving and reconstructing semantically meaningful visual content under severely constrained rates, ensuring perceptual plausibility and faithfulness to the scene intent. Specifically, the transmitter encodes the source image via JSCC and jointly transmits it with a textual prompt over the wireless channel. At the receiver, the corrupted low-rate representation is fused with the prompt and reconstructed through a Stable Diffusion model with ControlNet, enabling high-quality visual recovery. Leveraging both generative priors and semantic guidance, the proposed framework produces perceptually convincing images even under extreme bandwidth limitations. Experimental results demonstrate that the proposed method consistently outperforms conventional coding-based schemes and deep learning baselines, achieving superior perceptual quality and robustness across various channel conditions.
Paper Structure (13 sections, 19 equations, 8 figures, 1 algorithm)

This paper contains 13 sections, 19 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Overall framework
  4. JSCC-based Wireless Image Transmission
  5. The proposed strategy
  6. Latent JSCC-based Transmission
  7. Semantic Prompt Generation via MLLM
  8. Diffusion-Based Image Reconstruction
  9. Training Process
  10. Experiment Results
  11. Experimental Setup
  12. Experimental Results
  13. Conclusion

Figures (8)

  • Figure 1: Overview of the proposed system model.
  • Figure 2: The impact of semantic prompts on diffusion-based refinement.
  • Figure 3: Performance of different models on DIV2k dataset under AWGN and Rayleigh channel across different SNR. The CBR is set to 0.00326.
  • Figure 4: Performance of different models on Kodak dataset under AWGN and Rayleigh channel across different SNR. The CBR is set to 0.00326.
  • Figure 5: Performance of different models on DIV2k dataset under AWGN and Rayleigh channel across different CBRs. The SNR is set to 10 dB.
  • ...and 3 more figures