Take What You Need: Flexible Multi-Task Semantic Communications with Channel Adaptation

TL;DR

This work addresses the inefficiency of traditional data-centric transmission by introducing a channel-adaptive, multi-task semantic communication framework. It combines a ViT-based multi-task-aware scoring module, a CSI-driven channel-adaptive patch extractor, and a masked autoencoder backbone to transmit only semantically salient patches. By jointly optimizing semantic relevance and transmission efficiency, it demonstrates superior performance on image reconstruction and object detection under varying channel conditions, outperforming a random-masking MAE baseline. The proposed approach offers a scalable pathway for next-generation semantic networks that can handle diverse tasks across heterogeneous channels with reduced bandwidth through task-aware data prioritization.

Abstract

The growing demand for efficient semantic communication systems capable of managing diverse tasks and adapting to fluctuating channel conditions has driven the development of robust, resource-efficient frameworks. This article introduces a novel channel-adaptive and multi-task-aware semantic communication framework based on a masked auto-encoder architecture. Our framework optimizes the transmission of meaningful information by incorporating a multi-task-aware scoring mechanism that identifies and prioritizes semantically significant data across multiple concurrent tasks. A channel-aware extractor is employed to dynamically select relevant information in response to real-time channel conditions. By jointly optimizing semantic relevance and transmission efficiency, the framework ensures minimal performance degradation under resource constraints. Experimental results demonstrate the superior performance of our framework compared to conventional methods in tasks such as image reconstruction and object detection. These results underscore the framework's adaptability to heterogeneous channel environments and its scalability for multi-task applications, positioning it as a promising solution for next-generation semantic communication networks.

Figures (5)

• Figure 1: Illustration of the end-to-end channel-adaptive and multi-task-aware semantic communication framework.
• Figure 2: Network architecture details of the transceiver. In the Multi-Head Attention mechanism, $Q$ (Query) represents the feature vector corresponding to the current context or focus, $K$ (Key) acts as the reference information used for matching with the Query, and $V$ (Value) denotes the feature vector from which relevant information is extracted once a match is found.
• Figure 3: Visualization results of multi-task semantic communication under different mask ratio and different strategies.
• Figure 4: Image Reconstruction Performance Comparison under different SNR.
• Figure 5: Object Detection Performance Comparison under different SNR.