Robust Transmission of Punctured Text with Large Language Model-based Recovery

TL;DR

This work proposes a robust, data-agnostic text transmission scheme that punctures text by transmitting a subset of characters and recovers the complete text at the receiver using a pre-trained large language model (LLM), specifically GPT-3.5 Turbo. Central to the approach is the Importance Character Extractor (ICE), which, together with a random filters generator and a maximization-based filter selection, chooses the most informative characters to transmit under a compression ratio $\epsilon$, while signaling the chosen filter index $s$. The authors demonstrate that character-level omission with ICE-driven scoring significantly outperforms word-level omission and random filtering, and that the system exhibits strong robustness across datasets (Europarl and SQuAD) and tasks (text reconstruction and Q&A) even under low SNR, outperforming traditional bit-based communication on several metrics such as BLEU, sentence similarity, and Normalized Accuracy. By avoiding task-specific training and leveraging a pre-trained LLM for recovery, this method achieves higher data rates and improved resilience to channel distortions, with practical potential for real-world semantic communication.

Abstract

With the recent advancements in deep learning, semantic communication which transmits only task-oriented features, has rapidly emerged. However, since feature extraction relies on learning-based models, its performance fundamentally depends on the training dataset or tasks. For practical scenarios, it is essential to design a model that demonstrates robust performance regardless of dataset or tasks. In this correspondence, we propose a novel text transmission model that selects and transmits only a few characters and recovers the missing characters at the receiver using a large language model (LLM). Additionally, we propose a novel importance character extractor (ICE), which selects transmitted characters to enhance LLM recovery performance. Simulations demonstrate that the proposed filter selection by ICE outperforms random filter selection, which selects transmitted characters randomly. Moreover, the proposed model exhibits robust performance across different datasets and tasks and outperforms traditional bit-based communication in low signal-to-noise ratio conditions.

Figures (7)

• Figure 1: Comparison of the conventional semantic communication model and the proposed model.
• Figure 2: Example of ICE process applied to the word 'summer'.
• Figure 3: Example of Algorithm 1 applied to the word 'summer'.
• Figure 4: Performance comparison of word omission and character omission, where the same number of characters are omitted, under different remaining word ratios in a noiseless environment.
• Figure 5: Performance comparison of the proposed filter selection (solid lines) and random filter selection (dashed lines) under different compression ratios, $\epsilon$, in a noiseless environment, where $M$ denotes the number of filters.