Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variable-Length Joint Source-Channel Coding for Semantic Communication

Yujie Zhou, Rulong Wang, Yong Xiao, Yingyu Li, Guangming Shi

TL;DR

This work tackles the mismatch between continuous deep-JSCC encodings and discrete digital systems by introducing E2EC, an end-to-end framework that extends the information bottleneck to noisy channels and enables variable-length, discrete coding. By decomposing encoding into a length predictor and a content encoder, E2EC achieves bit-level rate control while maintaining semantic fidelity, using policy-gradient methods to train through non-differentiable steps. The transformed objective provides a computable bound via a variational cross-entropy, and the decoder employs a one-to-one embedding that maps binary codewords to semantically meaningful representations. Experiments on MNIST with a BSC show that E2EC outperforms fixed-length baselines and adapts its rate and content according to channel conditions, highlighting a practical path toward efficient, digital SemCom systems.

Abstract

This paper investigates a key challenge faced by joint source-channel coding (JSCC) in digital semantic communication (SemCom): the incompatibility between existing JSCC schemes that yield continuous encoded representations and digital systems that employ discrete variable-length codewords. It further results in feasibility issues in achieving physical bit-level rate control via such JSCC approaches for efficient semantic transmission. In this paper, we propose a novel end-to-end coding (E2EC) framework to tackle it. The semantic coding problem is formed by extending the information bottleneck (IB) theory over noisy channels, which is a tradeoff between bit-level communication rate and semantic distortion. With a structural decomposition of encoding to handle code length and content respectively, we can construct an end-to-end trainable encoder that supports the direct compression of a data source into a finite codebook. To optimize our E2EC across non-differentiable operations, e.g., sampling, we use the powerful policy gradient to support gradient-based updates. Experimental results illustrate that E2EC achieves high inference quality with low bit rates, outperforming representative baselines compatible with digital SemCom systems.

Variable-Length Joint Source-Channel Coding for Semantic Communication

TL;DR

This work tackles the mismatch between continuous deep-JSCC encodings and discrete digital systems by introducing E2EC, an end-to-end framework that extends the information bottleneck to noisy channels and enables variable-length, discrete coding. By decomposing encoding into a length predictor and a content encoder, E2EC achieves bit-level rate control while maintaining semantic fidelity, using policy-gradient methods to train through non-differentiable steps. The transformed objective provides a computable bound via a variational cross-entropy, and the decoder employs a one-to-one embedding that maps binary codewords to semantically meaningful representations. Experiments on MNIST with a BSC show that E2EC outperforms fixed-length baselines and adapts its rate and content according to channel conditions, highlighting a practical path toward efficient, digital SemCom systems.

Abstract

This paper investigates a key challenge faced by joint source-channel coding (JSCC) in digital semantic communication (SemCom): the incompatibility between existing JSCC schemes that yield continuous encoded representations and digital systems that employ discrete variable-length codewords. It further results in feasibility issues in achieving physical bit-level rate control via such JSCC approaches for efficient semantic transmission. In this paper, we propose a novel end-to-end coding (E2EC) framework to tackle it. The semantic coding problem is formed by extending the information bottleneck (IB) theory over noisy channels, which is a tradeoff between bit-level communication rate and semantic distortion. With a structural decomposition of encoding to handle code length and content respectively, we can construct an end-to-end trainable encoder that supports the direct compression of a data source into a finite codebook. To optimize our E2EC across non-differentiable operations, e.g., sampling, we use the powerful policy gradient to support gradient-based updates. Experimental results illustrate that E2EC achieves high inference quality with low bit rates, outperforming representative baselines compatible with digital SemCom systems.

Paper Structure

This paper contains 17 sections, 2 theorems, 13 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Notations
  4. System Model
  5. IB-induced Semantic Distortion over A Noisy Channel
  6. Problem Formulation
  7. The E2EC Framework
  8. Encoding with Structural Decomposition
  9. Decoding with One-to-One Embedding
  10. Transformed Objective
  11. Probability Parameterization
  12. Optimization
  13. Further Discussion
  14. Performance Evaluation
  15. Experimental Setup
  16. ...and 2 more sections

Key Result

Theorem 1

The following variational bound holds for SD: where $H(P_{Y|\hat{Z}}, Q_{{{Y}}|\hat{Z}}) \triangleq -\int p(y|\hat{z})\log {q(y|\hat{z})}dy$ is the conditional cross entropy. $p(y|\hat{z})$ is w.r.t. $P_{Y|\hat{Z}}$ and $q(y|\hat{z})$ is w.r.t. $Q_{Y|\hat{Z}}$. The equality holds iff $p(y|\hat{z}) = q(y|\hat{z})$.

Figures (2)

  • Figure 1: Illustration of a digital SemCom system. The channel is defined on bit sequences, absorbing digital modulation/demodulation mechanisms xie2023robust.
  • Figure 2: Numerical results: (a) The rate-semantic distortion tradeoff over a BSC. (b) The convergence of E2EC's $R$-$D$ curve on the information plane. (c) The evolution of $\mathrm{Var}(L)$, during training. (d) The non-normalized empirical $\hat{P}_L$, where frequency means the number of samples with the normalization constant 10,000 (the size of testing dataset). (e) The increment of $D$, i.e., $\Delta D$ varying by discarding equally divided bit-blocks. (f) The $l_2$-norm of 0-1 embeddings per bit, i.e., $\|e_1^i\|_2 = \|(1\times e_1^i+e_0^i)-(0\times e_1^i+e_0^i)\|_2, \forall i \in [R_{\max}]$.

Theorems & Definitions (8)

  • Definition 1
  • Remark 1
  • Theorem 1
  • proof
  • Corollary 1
  • proof
  • proof : Proof of Theorem \ref{['thm:vibo']}
  • proof : Proof of Corollary \ref{['cor:parambo']}