A Theory for Semantic Channel Coding With Many-to-one Source

TL;DR

The paper introduces a general semantic entropy that accounts for the value of symbols and the shared knowledge base, unifying prior task-oriented definitions and linking them to Shannon entropy. It then establishes a semantic channel coding theorem for many-to-one sources, proving achievability via joint typicality and a converse using a generalized Fano inequality that accommodates semantic mappings. The results yield a semantic capacity $C_{ ext{s}} = \max_{p(x)} \frac{I(X;Y)}{\alpha}$ and demonstrate that reliable semantic transmission is possible at rates below this bound, with a quantified impact of nonzero bit error rate on semantic accuracy. Numerical experiments on MNIST and CIFAR-10 validate the semantic entropy definition and the theorem, showing that high semantic accuracy can persist even when the physical channel operates below Shannon capacity, highlighting potential efficiency gains for future 6G semantic networks.

Abstract

As one of the potential key technologies of 6G, semantic communication is still in its infancy and there are many open problems, such as semantic entropy definition and semantic channel coding theory. To address these challenges, we investigate semantic information measures and semantic channel coding theorem. Specifically, we propose a semantic entropy definition as the uncertainty in the semantic interpretation of random variable symbols in the context of knowledge bases, which can be transformed into existing semantic entropy definitions under given conditions. Moreover, different from traditional communications, semantic communications can achieve accurate transmission of semantic information under a non-zero bit error rate. Based on this property, we derive a semantic channel coding theorem for a typical semantic communication with many-to-one source (i.e., multiple source sequences express the same meaning), and prove its achievability and converse based on a generalized Fano's inequality. Finally, numerical results verify the effectiveness of the proposed semantic entropy and semantic channel coding theorem.

Figures (6)

• Figure 1: A semantic communication system
• Figure 2: The "many-to-one" relations between ${U^n}$ and ${S^n}$.
• Figure 3: A task-oriented semantic communication system.
• Figure 4: Performance of the MNIST $N$-classification task
• Figure 5: Performance of the CIFAR10 $N$-classification task

Theorems & Definitions (4)

• Definition 1
• proof
• proof
• proof