Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Multi-User Semantic Communication via Transfer Learning and Knowledge Distillation

Loc X. Nguyen, Kitae Kim, Ye Lin Tun, Sheikh Salman Hassan, Yan Kyaw Tun, Zhu Han, Choong Seon Hong

TL;DR

The paper tackles multi-user semantic communication in a downlink setting where users have heterogeneous computing resources. It adopts a single base station encoder with Swin-transformer–based decoders at two users and introduces a two-stage training regimen that first trains with the high-capacity decoder, then freezes the encoder to train the low-capacity decoder, enhancing stability and speed. Two enhancements—partially transferring weights from the high- to low-capacity decoder and knowledge distillation from the high-capacity teacher to the low-capacity student—are proposed and evaluated. Experiments on the DIV2K dataset show improved reconstruction quality (PSNR) across SNRs and faster convergence, demonstrating scalable, resource-aware semantic communication for multi-user downlink networks. The practical impact lies in enabling efficient, high-quality semantic transmissions to devices with varying computing capabilities without requiring multiple encoders at the transmitter.

Abstract

Semantic communication, notable for ensuring quality of service by jointly optimizing source and channel coding, effectively extracts data semantics, reduces transmission length, and mitigates channel noise. However, most studies overlook multi-user scenarios and resource availability, limiting real-world application. This paper addresses this gap by focusing on downlink communication from a base station to multiple users with varying computing capacities. Users employ variants of Swin transformer models for source decoding and a simple architecture for channel decoding. We propose a novel training regimen, incorporating transfer learning and knowledge distillation to improve low-computing users' performance. Extensive simulations validate the proposed methods.

Optimizing Multi-User Semantic Communication via Transfer Learning and Knowledge Distillation

TL;DR

The paper tackles multi-user semantic communication in a downlink setting where users have heterogeneous computing resources. It adopts a single base station encoder with Swin-transformer–based decoders at two users and introduces a two-stage training regimen that first trains with the high-capacity decoder, then freezes the encoder to train the low-capacity decoder, enhancing stability and speed. Two enhancements—partially transferring weights from the high- to low-capacity decoder and knowledge distillation from the high-capacity teacher to the low-capacity student—are proposed and evaluated. Experiments on the DIV2K dataset show improved reconstruction quality (PSNR) across SNRs and faster convergence, demonstrating scalable, resource-aware semantic communication for multi-user downlink networks. The practical impact lies in enabling efficient, high-quality semantic transmissions to devices with varying computing capabilities without requiring multiple encoders at the transmitter.

Abstract

Semantic communication, notable for ensuring quality of service by jointly optimizing source and channel coding, effectively extracts data semantics, reduces transmission length, and mitigates channel noise. However, most studies overlook multi-user scenarios and resource availability, limiting real-world application. This paper addresses this gap by focusing on downlink communication from a base station to multiple users with varying computing capacities. Users employ variants of Swin transformer models for source decoding and a simple architecture for channel decoding. We propose a novel training regimen, incorporating transfer learning and knowledge distillation to improve low-computing users' performance. Extensive simulations validate the proposed methods.
Paper Structure (12 sections, 5 equations, 5 figures, 1 algorithm)

This paper contains 12 sections, 5 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model
  3. Multi-users in Semantic Communication
  4. Proposed Training Procedure
  5. Partially Transfer Learning between Users
  6. Knowledge Distillation from the Teacher to the Student
  7. Simulation Results
  8. Model and Dataset
  9. Benchmark
  10. Evaluating Convergence Speed of the Proposed Procedure for Low-computing Decoder
  11. Partially Transfer Learning and Knowledge Distillation Evaluation
  12. Conclusion

Figures (5)

  • Figure 1: The system model includes one transmitter and two receivers. The difference in the number of Swin-Transformer blocks leads to a difference in computing capacities between receivers.
  • Figure 2: The PSNR values of the proposed procedure, iterative training, and the train alone under SNR = 3dB.
  • Figure 3: The effects of transfer learning and knowledge distillation on the performance of LCD under SNR = 3dB.
  • Figure 4: The PSNR values of LCD after 500 epochs under different channel conditions.
  • Figure 5: The image reconstructed by different training methods under the SNR = 3dB.