Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Markov-Enforced Discrete Diffusion Model for Digital Semantic Symbol Error Correction

Yoon Huh, Jeongho Kang, Wan Choi

Abstract

Diffusion models (DMs) have achieved remarkable success across various domains owing to their strong generative and denoising capabilities. Meanwhile, semantic communication based on neural joint source-channel coding (JSCC) has emerged as a promising paradigm for robust and efficient image transmission. However, severe channel noise can still distort the transmitted semantic symbols, resulting in significant performance degradation. Applying DMs to digital semantic symbols, particularly in vector quantization (VQ)-based systems, is fundamentally challenging because the Markov assumption does not hold for the symbol transition dynamics. To address this issue, we introduce SSCDM, a semantic symbol correcting diffusion model whose discrete-time transition dynamics are constructed using solutions from continuous-time Markov chain theory. Furthermore, to promote synergy between DMs and JSCC, our DM structure embeds discrete symbols into a latent feature space using a learned VQ codebook, and a self-organizing map-based loss is incorporated during codebook learning to enhance the geometric vicinity between neighboring digital symbols, thereby promoting topology-preserving semantic representations. Experimental results show that the proposed method significantly improves image reconstruction quality and outperforms previous symbol-level denoising techniques under low signal-to-noise ratio scenarios and different datasets.

Markov-Enforced Discrete Diffusion Model for Digital Semantic Symbol Error Correction

Abstract

Diffusion models (DMs) have achieved remarkable success across various domains owing to their strong generative and denoising capabilities. Meanwhile, semantic communication based on neural joint source-channel coding (JSCC) has emerged as a promising paradigm for robust and efficient image transmission. However, severe channel noise can still distort the transmitted semantic symbols, resulting in significant performance degradation. Applying DMs to digital semantic symbols, particularly in vector quantization (VQ)-based systems, is fundamentally challenging because the Markov assumption does not hold for the symbol transition dynamics. To address this issue, we introduce SSCDM, a semantic symbol correcting diffusion model whose discrete-time transition dynamics are constructed using solutions from continuous-time Markov chain theory. Furthermore, to promote synergy between DMs and JSCC, our DM structure embeds discrete symbols into a latent feature space using a learned VQ codebook, and a self-organizing map-based loss is incorporated during codebook learning to enhance the geometric vicinity between neighboring digital symbols, thereby promoting topology-preserving semantic representations. Experimental results show that the proposed method significantly improves image reconstruction quality and outperforms previous symbol-level denoising techniques under low signal-to-noise ratio scenarios and different datasets.
Paper Structure (23 sections, 5 theorems, 23 equations, 9 figures, 3 algorithms)

This paper contains 23 sections, 5 theorems, 23 equations, 9 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Discrete Diffusion Model
  4. Continuous-Time Markov Chain
  5. System Model
  6. Semantic Symbol Error Correcting Discrete Diffusion Model
  7. Discussions on Transition Matrix Design
  8. Markov-Enforced Transition Matrices
  9. Semantics-Informed Diffusion Model Architecture
  10. Training and Inference Algorithms
  11. Training Method
  12. SOM-based Codebook Learning
  13. Decoding Process with SSCDM
  14. Experimental Results
  15. Transition Matrix Similarity
  16. ...and 8 more sections

Key Result

Proposition 1

For the CTMC forward rate matrix, it holds that

Figures (9)

  • Figure 1: An illustration of (a) vector quantization ($M = 16$), (b) constellation symbol mapping ($M = 16$), (c) forward and reverse diffusion processes ($M = 4$), and (d) the design philosophy of the proposed discrete diffusion model ($M = 4$). Note that the value of $M$ differs in (a)–(d) for visualization purposes only, while it should remain consistent throughout the actual system.
  • Figure 2: Diagonal entries of $\{\mathbf{D}(t_k)\}_{k = 0}^T$ obtained via spline interpolation from the optimized subsampled matrices $\{\mathbf{D}(t_{k_\ell})\}_{\ell = 1}^{T'}$ when $M = 16$.
  • Figure 3: Heatmaps of 16-QAM transition matrix for the ground-truth, SSCDM, and DCDDM under different SNR conditions.
  • Figure 4: NMSE of the designed transition matrices of SSCDM and DCDDM.
  • Figure 5: MS-SSIM$\uparrow$ (orange) and LPIPS$\downarrow$ (blue) comparison between SSCDM and baseline methods on (a) FFHQ and (b) CelebA datasets.
  • ...and 4 more figures

Theorems & Definitions (7)

  • Proposition 1: Rate matrix properties
  • Proposition 2: Transition matrix condition
  • Theorem 1
  • Remark 1
  • Remark 2
  • Lemma 1
  • Lemma 2