Context Video Semantic Transmission with Variable Length and Rate Coding over MIMO Channels

TL;DR

This work tackles semantic video transmission over MIMO channels by integrating contextual video features with CSI-aware, variable length and rate coding. The proposed CVST framework introduces a context-channel correlation map, a multi-reference variable length and rate coding (MR-VLRC) scheme with checkerboard entropy coding, and a checkerboard-based feature modulation to support multiple rate points within a single trained model. The approach demonstrates substantial gains over DL-based and traditional SSCC baselines, showing improved PSNR, MS-SSIM, and LPIPS across varying SNRs, CBRs, and channel conditions, including imperfect CSI and 3GPP CDL channels. These results indicate CVST's potential for robust, flexible, and deployment-friendly wireless video transmission in real-world MIMO environments.

Abstract

The evolution of semantic communications has profoundly impacted wireless video transmission, whose applications dominate driver of modern bandwidth consumption. However, most existing schemes are predominantly optimized for simple additive white Gaussian noise or Rayleigh fading channels, neglecting the ubiquitous multiple-input multiple-output (MIMO) environments that critically hinder practical deployment. To bridge this gap, we propose the context video semantic transmission (CVST) framework under MIMO channels. Building upon an efficient contextual video transmission backbone, CVST effectively learns a context-channel correlation map to explicitly formulate the relationships between feature groups and MIMO subchannels. Leveraging these channel-aware features, we design a multi-reference entropy coding mechanism, enabling channel state-aware variable length coding. Furthermore, CVST incorporates a checkerboard-based feature modulation strategy to achieve multiple rate points within a single trained model, thereby enhancing deployment flexibility. These innovations constitute our multi-reference variable length and rate coding (MR-VLRC) scheme. By integrating contextual transmission with MR-VLRC, CVST demonstrates substantial performance gains over various standardized separated coding methods and recent wireless video semantic communication approaches. The code is available at https://github.com/xie233333/CVST.

Figures (13)

• Figure 1: The proposed CVST framework. The red line is the context-based semantic feature transmission link while the blue line is the motion vector transmission link. The dashed line refers to the MIMO CSI feedback link.
• Figure 2: The context-channel correlation map for matching the context channel group and MIMO subchannel pairs.
• Figure 3: The proposed multi-reference variable rate and length entropy coding. The entropy coding is divided into anchored and non-anchored parts. For a specific feature channel group, the anchored part is first learned with multi-references embeddings. Then, the non-anchored part is learned. $\eta_{t,i}^c$ is adjustable according to the CSI-aware rate allocation.
• Figure 4: (a) The structure of motion vector/context VLRC coder including checkerboard feature modulation, VLR entropy coding, multi-references fusion coder, and rate adaptation coder. (b) The structure of multi-reference fusion motion vector/context coder.
• Figure 5: The network structures of different modules. (a) Context generator; (b) Frame regeneration; (c) Frame refinement.