DiT-JSCC: Rethinking Deep JSCC with Diffusion Transformers and Semantic Representations

TL;DR

DiT-JSCC tackles semantic fidelity in image transmission over extreme wireless channels by rethinking what information should be sent. It introduces a semantics-prioritized dual-branch encoder that encodes high-level semantic content and residual details, paired with a coarse-to-fine diffusion-transformer decoder that jointly refines global structure and textures. A KC-inspired bandwidth allocation strategy further optimizes per-image semantic bandwidth during transmission. The approach outperforms existing JSCC and diffusion-based methods in semantic consistency and perceptual realism across AWGN and Rayleigh channels, highlighting the practical value of explicit semantic guidance for generative decoding.

Abstract

Generative joint source-channel coding (GJSCC) has emerged as a new Deep JSCC paradigm for achieving high-fidelity and robust image transmission under extreme wireless channel conditions, such as ultra-low bandwidth and low signal-to-noise ratio. Recent studies commonly adopt diffusion models as generative decoders, but they frequently produce visually realistic results with limited semantic consistency. This limitation stems from a fundamental mismatch between reconstruction-oriented JSCC encoders and generative decoders, as the former lack explicit semantic discriminability and fail to provide reliable conditional cues. In this paper, we propose DiT-JSCC, a novel GJSCC backbone that can jointly learn a semantics-prioritized representation encoder and a diffusion transformer (DiT) based generative decoder, our open-source project aims to promote the future research in GJSCC. Specifically, we design a semantics-detail dual-branch encoder that aligns naturally with a coarse-to-fine conditional DiT decoder, prioritizing semantic consistency under extreme channel conditions. Moreover, a training-free adaptive bandwidth allocation strategy inspired by Kolmogorov complexity is introduced to further improve the transmission efficiency, thereby indeed redefining the notion of information value in the era of generative decoding. Extensive experiments demonstrate that DiT-JSCC consistently outperforms existing JSCC methods in both semantic consistency and visual quality, particularly in extreme regimes.

Figures (14)

• Figure 1: Semantic consistency is more critical than pixel-level consistency for human perception under extremely limited bandwidth or low SNR conditions. SNR = 0 and CBR = 1/96 for all methods. Visually, our method produces the most faithful reconstructions, both realistic and semantically aligned with the original. In terms of metrics, our approach achieves the best performance across all semantic consistency metrics (LPIPS lpips, DISTS dists, Dreamsim fu2023learning, CLIP radford2021learning, DINOv2 oquab2023dinov2), while its PSNR is much lower than the original deep JSCC. Notably, only our method successfully captures the original textual semantics. * denotes a score inversion operation, for example, $\text{LPIPS*} = 1 - \text{LPIPS}$.
• Figure 2: Overall framework of the proposed DiT-JSCC (d) and comparison with the original deep JSCC (a) and other existing Generative JSCC structures (b and c).
• Figure 3: Overview of our DiT-JSCC system architecture.
• Figure 4: The representations learned from different branches exhibit distinct characteristics: the semantic branch focuses on structurally regular and semantically rich objects, while the detail branch targets high-frequency regions with details.
• Figure 5: Reconstruction results under different bandwidth allocations of the two branches. When only the semantic signal is available ($\rho _s>0, \rho _d=0$), the reconstruction retains the overall semantic structure but appears blurry details. In contrast, when only the detail signal is transmitted ($\rho _s=0, \rho _d>0$), the model fails to produce meaningful reconstructions.