Feature Importance-Aware Deep Joint Source-Channel Coding for Computationally Efficient and Adjustable Image Transmission

TL;DR

FAJSCC tackles the high computational cost of deepJSCC for image transmission by introducing axis-dimension specialized computation and selective deformable self-attention, guided by feature importance and organized via an attention family tree. It enables independent adjustment of encoder and decoder complexity, maintaining high transmission quality under AWGN and fading channels with lower FLOPs and memory than state-of-the-art models. Key findings show the decoder’s perception of noisy signals drives the most computation, and the proposed framework consistently outperforms baselines across CPP and SNR settings. This work offers a practical path toward deployment of deepJSCC in resource-constrained vision IoT applications.

Abstract

Recent advances in deep learning-based joint source-channel coding (deepJSCC) have substantially improved communication performance, but their high computational cost hinders practical deployment. Moreover, certain applications require the ability to dynamically adapt computational complexity. To address these issues, we propose a Feature Importance-Aware deepJSCC (FAJSCC) model for image transmission that is both computationally efficient and adjustable. FAJSCC employs axis-dimension specialized computation, which performs efficient operations individually for each spatial and channel axis, significantly reducing computational cost while representing features effectively. It further incorporates selective deformable self-attention, which applies self-attention only to selected and adaptively adjusted features, leveraging the importance and relations of input features to efficiently capture complex feature correlations. Another key feature of FAJSCC is that the number of selected important areas can be controlled separately by the encoder and the decoder, depending on the available computational budget. It makes FAJSCC the first deepJSCC architecture to allow independent adjustment of encoder and decoder complexity within a single trained model. Experimental results show that FAJSCC achieves superior image transmission performance under various channel conditions while requiring less computational complexity than recent state-of-the-art models. Furthermore, experiments independently varying the encoder and decoder's computational resources reveal, for the first time in the deepJSCC literature, that understanding the meaning of noisy features in the decoder demands the greatest computational cost. The code is publicly available at github.com/hansung-choi/FAJSCCv2.

Figures (10)

• Figure 1: A point-to-point deepJSCC communication system.
• Figure 2: FAJSCC Architecture.
• Figure 3: Dimension specialized operations of the proposed FA block.
• Figure 4: Visualization of feature deformation.
• Figure 5: Procedures of selective deformable self-attention.