Separate Source Channel Coding Is Still What You Need: An LLM-based Rethinking

TL;DR

This work reevaluates the SemCom landscape by advocating Separate Source Channel Coding (SSCC) as a viable and often superior approach to Joint Source Channel Coding (JSCC) for text transmission. It couples LLM-based arithmetic coding for highly efficient text compression with ECCT, a transformer-inspired channel decoder that predicts multiplicative channel noise, forming a robust SSCC pipeline. Across AWGN and Rayleigh channels, the method achieves higher word-level BLEU and semantic similarity than competitive JSCC schemes, while also highlighting energy costs of high-precision float transmission that can undermine JSCC gains. The results suggest SSCC with LLM-AC and ECCT not only delivers strong performance but also offers a practical, energy-aware framework for modern communication systems handling semantic content, guiding future research toward scalable, multimodal extensions and efficient model design. $SNR$ assessments and compression-rate analyses reinforce the practical benefits and tradeoffs of separation over end-to-end learning in this context.

Abstract

Along with the proliferating research interest in Semantic Communication (SemCom), Joint Source Channel Coding (JSCC) has dominated the attention due to the widely assumed existence in efficiently delivering information semantics. Nevertheless, this paper challenges the conventional JSCC paradigm, and advocates for adoption of Separate Source Channel Coding (SSCC) to enjoy the underlying more degree of freedom for optimization. We demonstrate that SSCC, after leveraging the strengths of Large Language Model (LLM) for source coding and Error Correction Code Transformer (ECCT) complemented for channel decoding, offers superior performance over JSCC. Our proposed framework also effectively highlights the compatibility challenges between SemCom approaches and digital communication systems, particularly concerning the resource costs associated with the transmission of high precision floating point numbers. Through comprehensive evaluations, we establish that empowered by LLM-based compression and ECCT-enhanced error correction, SSCC remains a viable and effective solution for modern communication systems. In other words, separate source and channel coding is still what we need!

Figures (12)

• Figure 1: Framework of LLM-based and ECCT-complemented SSCC system.
• Figure 2: An example of arithmetic coding.
• Figure 3: LLM-based arithmetic encoding and decoding.
• Figure 4: ECCT architecture.
• Figure 5: BLEU and Similarity scores versus $\textbf{SNR}_{\textbf{unified}}$ are evaluated for the same number of transmitted symbols. The proposed LLM-based SSCC is compared with Huffman coding with $\text{LDPC}(49,24)$ in BPSK, DeepSC, UT, and UT with quantization under the AWGN channel.

Theorems & Definitions (2)

• Remark 1
• Remark 2