Hierarchy-Aware and Channel-Adaptive Semantic Communication for Bandwidth-Limited Data Fusion

TL;DR

This work tackles bandwidth-limited hyperspectral super-resolution by fusing LR-HSI and HR-RGB through a hierarchy-aware semantic communication framework. It introduces a channel-adaptive Transformer-based fusion that jointly preserves deep structural information and shallow details without increasing transmitted data, achieving up to 2 dB PSNR gain and around two-thirds bandwidth reduction on the CAVE and DC Mall datasets. The approach comprises dedicated spectral and spatial feature extractors, a hierarchy-aware fusion module, and a reconstruction decoder, with rigorous ablations showing robustness to channel conditions and different feature dimensions. The results demonstrate practical impact for bandwidth-constrained remote sensing, enabling high-quality HR-HSI reconstruction in real-time or resource-limited scenarios.

Abstract

Obtaining high-resolution hyperspectral images (HR-HSI) is costly and data-intensive, making it necessary to fuse low-resolution hyperspectral images (LR-HSI) with high-resolution RGB images (HR-RGB) for practical applications. However, traditional fusion techniques, which integrate detailed information into the reconstruction, significantly increase bandwidth consumption compared to directly transmitting raw data. To overcome these challenges, we propose a hierarchy-aware and channel-adaptive semantic communication approach for bandwidth-limited data fusion. A hierarchical correlation module is proposed to preserve both the overall structural information and the details of the image required for super-resolution. This module efficiently combines deep semantic and shallow features from LR-HSI and HR-RGB. To further reduce bandwidth usage while preserving reconstruction quality, a channel-adaptive attention mechanism based on Transformer is proposed to dynamically integrate and transmit the deep and shallow features, enabling efficient data transmission and high-quality HR-HSI reconstruction. Experimental results on the CAVE and Washington DC Mall datasets demonstrate that our method outperforms single-source transmission, achieving up to a 2 dB improvement in peak signal-to-noise ratio (PSNR). Additionally, it reduces bandwidth consumption by two-thirds, confirming its effectiveness in bandwidth-constrained environments for HR-HSI reconstruction tasks.

Figures (7)

• Figure 1: The proposed hierarchy-aware and channel-adaptive semantic communication architecture for bandwidth-limited data fusion.
• Figure 2: The detailed structure of the proposed fused feature extraction module at the transmitter. $F_{{cr}^{*}}\left(\cdot\right)$ is represented as $\text{CR}^{*}$. $\bigoplus$ and $\bigotimes$ denote addition and multiplication of the same position elements, respectively.
• Figure 3: The detailed structure of the proposed adapative fusion block in the hierarchy-aware fusion module.
• Figure 4: The structure of the fusion decoder. $F_{{ctr}^{3}}\left(\cdot\right)$ is represented as $\text{CTR}^{3}$
• Figure 5: Performance comparisons on the CAVE and DC dataset.