Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation

Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, Wenjun Zhang

TL;DR

This work addresses the mismatch between conventional rate-distortion-based communication and human perception by reframing transmission as a joint source-channel-generation problem. It introduces Joint Source-Channel-Generation Coding (JSCGC), where a generator at the receiver samples from the conditional distribution $p(\mathbf{x}|\hat{\mathbf{y}})$ to produce perceptually faithful outputs, while the encoder and channel are optimized to maximize $I(\mathbf{X}; \hat{\mathbf{Y}})$ under a perceptual constraint $d_p(p(\mathbf{X}), p(G_\theta(\hat{\mathbf{Y}}))) \le \zeta$. A variational diffusion-based training objective is derived, along with a diffusion-reverse sampling algorithm and a theoretical lower bound on the maximum semantic inconsistency $\delta^*$ as a function of $I(\mathbf{X}; \hat{\mathbf{Y}})$ and the intrinsic dimension $d$ of the data manifold. Experiments on image transmission show that JSCGC improves perceptual quality and semantic fidelity over distortion-oriented JSCC methods, revealing a fundamental shift in how errors manifest in generative communications and offering a practical path to semantically faithful communication under bandwidth constraints.

Abstract

Conventional communication systems, including both separation-based coding and AI-driven joint source-channel coding (JSCC), are largely guided by Shannon's rate-distortion theory. However, relying on generic distortion metrics fails to capture complex human visual perception, often resulting in blurred or unrealistic reconstructions. In this paper, we propose Joint Source-Channel-Generation Coding (JSCGC), a novel paradigm that shifts the focus from deterministic reconstruction to probabilistic generation. JSCGC leverages a generative model at the receiver as a generator rather than a conventional decoder to parameterize the data distribution, enabling direct maximization of mutual information under channel constraints while controlling stochastic sampling to produce outputs residing on the authentic data manifold with high fidelity. We further derive a theoretical lower bound on the maximum semantic inconsistency with given transmitted mutual information, elucidating the fundamental limits of communication in controlling the generative process. Extensive experiments on image transmission demonstrate that JSCGC substantially improves perceptual quality and semantic fidelity, significantly outperforming conventional distortion-oriented JSCC methods.

Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation

TL;DR

This work addresses the mismatch between conventional rate-distortion-based communication and human perception by reframing transmission as a joint source-channel-generation problem. It introduces Joint Source-Channel-Generation Coding (JSCGC), where a generator at the receiver samples from the conditional distribution to produce perceptually faithful outputs, while the encoder and channel are optimized to maximize under a perceptual constraint . A variational diffusion-based training objective is derived, along with a diffusion-reverse sampling algorithm and a theoretical lower bound on the maximum semantic inconsistency as a function of and the intrinsic dimension of the data manifold. Experiments on image transmission show that JSCGC improves perceptual quality and semantic fidelity over distortion-oriented JSCC methods, revealing a fundamental shift in how errors manifest in generative communications and offering a practical path to semantically faithful communication under bandwidth constraints.

Abstract

Conventional communication systems, including both separation-based coding and AI-driven joint source-channel coding (JSCC), are largely guided by Shannon's rate-distortion theory. However, relying on generic distortion metrics fails to capture complex human visual perception, often resulting in blurred or unrealistic reconstructions. In this paper, we propose Joint Source-Channel-Generation Coding (JSCGC), a novel paradigm that shifts the focus from deterministic reconstruction to probabilistic generation. JSCGC leverages a generative model at the receiver as a generator rather than a conventional decoder to parameterize the data distribution, enabling direct maximization of mutual information under channel constraints while controlling stochastic sampling to produce outputs residing on the authentic data manifold with high fidelity. We further derive a theoretical lower bound on the maximum semantic inconsistency with given transmitted mutual information, elucidating the fundamental limits of communication in controlling the generative process. Extensive experiments on image transmission demonstrate that JSCGC substantially improves perceptual quality and semantic fidelity, significantly outperforming conventional distortion-oriented JSCC methods.
Paper Structure (11 sections, 4 theorems, 22 equations, 4 figures)

This paper contains 11 sections, 4 theorems, 22 equations, 4 figures.

Table of Contents

  1. Introduction
  2. System Model
  3. Proposed JSCGC Scheme
  4. JSCGC Training Algorithm
  5. Sampling Algorithm of the JSCGC
  6. Theoretical Bound on Maximum Semantic Inconsistency
  7. Implementation and Experimental Results
  8. Architecture Design
  9. Experimental Setup
  10. Results
  11. Conclusion

Key Result

Proposition 1

The conditional entropy $H(\mathbf{X}|\hat{\mathbf{Y}})$ can be minimized via the following gradient-based loss: where $\mathbf{x}_0 \sim p(\mathbf{x})$, $\mathbf{x}_T \sim N(0,\mathbf{I})$, $\mathbf{x}_t=(1-\frac{t}{T})\mathbf{x}_0+\frac{t}{T} \mathbf{x}_T$ for $t\in\{1,2,...,T\}$, and $\mathbf{\nu}_\theta(\cdot)$ is the output of the neural network in the generator.

Figures (4)

  • Figure 1: System model: (a) reconstruction paradigm; (b) generation paradigm.
  • Figure 2: An implementation of the proposed JSCGC scheme.
  • Figure 3: Visualization results under the AWGN channel.
  • Figure 4: The performance of different schemes versus SNR under AWGN channel with CBR = $\frac{1}{384}$. (a) PSNR. (b) LPIPS. (c) DINO score. (d) rFID.

Theorems & Definitions (11)

  • Remark 1
  • Proposition 1
  • proof
  • Lemma 1
  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • proof
  • Corollary 1
  • ...and 1 more