Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Short Wins Long: Short Codes with Language Model Semantic Correction Outperform Long Codes

Jiafu Hao, Chentao Yue, Hao Chang, Branka Vucetic, Yonghui Li

TL;DR

The paper tackles URLLC-style transmission of natural language by decomposing a sentence into multiple short block codes and applying a BART-based semantic error correction to recover semantically corrupted content. The approach, labeled MSC, leverages parallel decoding of short BCH blocks and context-aware reconstruction to surpass a single long LDPC code in BLER and semantic metrics, while also reducing latency. A semantic-confidence HARQ (SemConf HARQ) extension selectively retransmits the most semantically uncertain segments, yielding additional gains in BLER and BLEU/ROUGE-L with modest overhead. Experimental results on SNLI-derived data show up to 0.7 dB BLER improvement and substantial semantic fidelity gains, demonstrating practical potential for meaning-aware communications with short codes and LM-based correction.

Abstract

This paper presents a novel semantic-enhanced decoding scheme for transmitting natural language sentences with multiple short block codes over noisy wireless channels. After ASCII source coding, the natural language sentence message is divided into segments, where each is encoded with short block channel codes independently before transmission. At the receiver, each short block of codewords is decoded in parallel, followed by a semantic error correction (SEC) model to reconstruct corrupted segments semantically. We design and train the SEC model based on Bidirectional and Auto-Regressive Transformers (BART). Simulations demonstrate that the proposed scheme can significantly outperform encoding the sentence with one conventional long LDPC code, in terms of block error rate (BLER), semantic metrics, and decoding latency. Finally, we proposed a semantic hybrid automatic repeat request (HARQ) scheme to further enhance the error performance, which selectively requests retransmission depends on semantic uncertainty.

Short Wins Long: Short Codes with Language Model Semantic Correction Outperform Long Codes

TL;DR

The paper tackles URLLC-style transmission of natural language by decomposing a sentence into multiple short block codes and applying a BART-based semantic error correction to recover semantically corrupted content. The approach, labeled MSC, leverages parallel decoding of short BCH blocks and context-aware reconstruction to surpass a single long LDPC code in BLER and semantic metrics, while also reducing latency. A semantic-confidence HARQ (SemConf HARQ) extension selectively retransmits the most semantically uncertain segments, yielding additional gains in BLER and BLEU/ROUGE-L with modest overhead. Experimental results on SNLI-derived data show up to 0.7 dB BLER improvement and substantial semantic fidelity gains, demonstrating practical potential for meaning-aware communications with short codes and LM-based correction.

Abstract

This paper presents a novel semantic-enhanced decoding scheme for transmitting natural language sentences with multiple short block codes over noisy wireless channels. After ASCII source coding, the natural language sentence message is divided into segments, where each is encoded with short block channel codes independently before transmission. At the receiver, each short block of codewords is decoded in parallel, followed by a semantic error correction (SEC) model to reconstruct corrupted segments semantically. We design and train the SEC model based on Bidirectional and Auto-Regressive Transformers (BART). Simulations demonstrate that the proposed scheme can significantly outperform encoding the sentence with one conventional long LDPC code, in terms of block error rate (BLER), semantic metrics, and decoding latency. Finally, we proposed a semantic hybrid automatic repeat request (HARQ) scheme to further enhance the error performance, which selectively requests retransmission depends on semantic uncertainty.
Paper Structure (20 sections, 10 equations, 3 figures, 4 tables)

This paper contains 20 sections, 10 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Linear block code
  4. Pretrained Language Models
  5. Proposed Semantic Error Correction Scheme
  6. System Overview
  7. Semantic Error Correction (SEC)
  8. Semantic Confidence HARQ
  9. Experimental Results and Discussion
  10. Experimental Setup
  11. Dataset
  12. Implementation Configuration
  13. Evaluation metrics
  14. Performance of the proposed approach with SEC
  15. BLER Performance
  16. ...and 5 more sections

Figures (3)

  • Figure 1: Proposed MSC approach using multiple short block codes segments and SEC. The modulation and demodulation are omitted for clarity.
  • Figure 2: Performance of proposed MSC scheme comapred to TLC scheme in BLER, BLEU, and ROUGE-L.
  • Figure 3: Performance of proposed proposed MSC scheme with SemConf HARQ in BLER, BLEU, and ROUGE-L.