Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Importance-Aware Communications Using Pre-trained Language Models

Shuaishuai Guo, Yanhu Wang, Shujing Li, Nasir Saeed

TL;DR

This work addresses semantic communications by quantifying frame semantic importance with pre-trained language models and steering cross-layer resource allocation. It introduces ChatGPT-SIAC and BERT-SIAC, two SIAC schemes that embed a cross-layer manager to derive frame importance and guide priority-based transmission. A semantic-importance-aware power allocation formulation is developed, deriving frame outage probabilities under Rayleigh fading and solving a constrained optimization to minimize semantic loss, with practical implementation via Manopt. Experimental results on the EuroParl corpus show that SIAC outperforms equal-priority transmissions, with ChatGPT-SIAC excelling in word-level reliability and BERT-SIAC in semantic-loss metrics, while highlighting latency considerations for different deployment strategies.

Abstract

This letter proposes a semantic importance-aware communication (SIAC) scheme using pre-trained language models (e.g., ChatGPT, BERT, etc.). Specifically, we propose a cross-layer design with a pre-trained language model embedded in/connected by the cross-layer manager. The pre-trained language model is utilized to quantify the semantic importance of data frames. Based on the quantified semantic importance, we investigate semantic importance-aware power allocation. Unlike existing deep joint source-channel coding (Deep-JSCC)-based semantic communication schemes, SIAC can be directly embedded into current communication systems by only introducing a cross-layer manager. Our experimental results show that the proposed SIAC scheme can achieve lower semantic loss than existing equal-priority communications.

Semantic Importance-Aware Communications Using Pre-trained Language Models

TL;DR

This work addresses semantic communications by quantifying frame semantic importance with pre-trained language models and steering cross-layer resource allocation. It introduces ChatGPT-SIAC and BERT-SIAC, two SIAC schemes that embed a cross-layer manager to derive frame importance and guide priority-based transmission. A semantic-importance-aware power allocation formulation is developed, deriving frame outage probabilities under Rayleigh fading and solving a constrained optimization to minimize semantic loss, with practical implementation via Manopt. Experimental results on the EuroParl corpus show that SIAC outperforms equal-priority transmissions, with ChatGPT-SIAC excelling in word-level reliability and BERT-SIAC in semantic-loss metrics, while highlighting latency considerations for different deployment strategies.

Abstract

This letter proposes a semantic importance-aware communication (SIAC) scheme using pre-trained language models (e.g., ChatGPT, BERT, etc.). Specifically, we propose a cross-layer design with a pre-trained language model embedded in/connected by the cross-layer manager. The pre-trained language model is utilized to quantify the semantic importance of data frames. Based on the quantified semantic importance, we investigate semantic importance-aware power allocation. Unlike existing deep joint source-channel coding (Deep-JSCC)-based semantic communication schemes, SIAC can be directly embedded into current communication systems by only introducing a cross-layer manager. Our experimental results show that the proposed SIAC scheme can achieve lower semantic loss than existing equal-priority communications.
Paper Structure (12 sections, 4 equations, 4 figures, 2 tables)

This paper contains 12 sections, 4 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System Model
  3. Semantic Importance Quantification
  4. Using Generative Language Models Like ChatGPT
  5. Using Discriminative Language Models Like BERT
  6. Overhead Analysis
  7. Communication content is generated by generative language models like ChatGPT
  8. Communication content has already been pre-generated by other sources
  9. Communication content is generated by other sources in real-time
  10. Semantic Importance-Aware Power Allocation
  11. Experiment Results and Discussions
  12. Conclusions

Figures (4)

  • Figure 1: Demonstration of SIAC systems, which target to reliably transmit the semantic meaning in data frames.
  • Figure 2: SIAC cross-layer structure with pre-trained language model (e.g., ChatGPT, BERT, etc.) embedded in/connected by the cross-layer manager.
  • Figure 3: Comparisons among ChatGPT-SIAC, BERT-SIAC, and equal-priority transmission.
  • Figure 4: Empirical study on the word generation speed of ChatGPT, frame importance quantification time of self-hosted BERT and ChatGPT.