Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Block Erasure-Aware Semantic Multimedia Compression via JSCC Autoencoder

Homa Esfahanizadeh, Nargis Fayaz, Jinfeng Du, Harish Viswanathan

TL;DR

An AI-based framework for semantic transmission of multimedia data over band-limited, time-varying channels achieves reliable semantic reconstruction with graceful quality degradation as channel conditions worsen, eliminating the need for retransmissions that cause unacceptable delays in latency-sensitive applications such as video conferencing and robotic control.

Abstract

We present an AI-based framework for semantic transmission of multimedia data over band-limited, time-varying channels. The method targets scenarios where large content is split into multiple packets, with an unknown number potentially dropped due to channel impairments. Using joint source-channel coding (JSCC), our approach achieves reliable semantic reconstruction with graceful quality degradation as channel conditions worsen, eliminating the need for retransmissions that cause unacceptable delays in latency-sensitive applications such as video conferencing and robotic control. The framework is compatible with existing network protocols and further enables intelligent congestion control and unequal error protection. A tunable design parameter allows balancing robustness at low channel quality against fidelity at high channel quality. Experiments demonstrate significant robustness improvement over state-of-the-art baselines in both image and video domains.

Block Erasure-Aware Semantic Multimedia Compression via JSCC Autoencoder

TL;DR

An AI-based framework for semantic transmission of multimedia data over band-limited, time-varying channels achieves reliable semantic reconstruction with graceful quality degradation as channel conditions worsen, eliminating the need for retransmissions that cause unacceptable delays in latency-sensitive applications such as video conferencing and robotic control.

Abstract

We present an AI-based framework for semantic transmission of multimedia data over band-limited, time-varying channels. The method targets scenarios where large content is split into multiple packets, with an unknown number potentially dropped due to channel impairments. Using joint source-channel coding (JSCC), our approach achieves reliable semantic reconstruction with graceful quality degradation as channel conditions worsen, eliminating the need for retransmissions that cause unacceptable delays in latency-sensitive applications such as video conferencing and robotic control. The framework is compatible with existing network protocols and further enables intelligent congestion control and unequal error protection. A tunable design parameter allows balancing robustness at low channel quality against fidelity at high channel quality. Experiments demonstrate significant robustness improvement over state-of-the-art baselines in both image and video domains.
Paper Structure (14 sections, 2 equations, 12 figures)

This paper contains 14 sections, 2 equations, 12 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Joint source channel coding (JSCC)
  4. Successive refinement
  5. In-painting
  6. Rateless compression using dropouts
  7. Method
  8. Code design using JSCC trained for block erasure channel
  9. Tunable trade-offs across network operating regimes
  10. Applications: Semantic network operations that can benefit from the proposed source code
  11. Experiments
  12. Image
  13. Video
  14. Conclusion

Figures (12)

  • Figure 1: Our encoder converts the multimedia sample into blocks with predefined importance levels. Here, darker colors indicate blocks that are more important for reconstruction. Depending on the bandwidth and channel quality, the network delivers a subset of blocks to the decoder, where the original sample is approximately reconstructed from the received subset.
  • Figure 2: Designing the JSCC using a multi-level block erasure abstracted channel, enabling intelligent error concealment.
  • Figure 3: Architecture of the encoder and decoder.
  • Figure 4: Performance of the proposed method trained under various uniform erasure probabilities and evaluated across different bandwidths.
  • Figure 5: Mismatch analysis showing the average PSNR for codes trained with different $\epsilon_\text{train}$ and tested under varying $\epsilon_\text{test}$. Vertical lines indicate matched training and test conditions.
  • ...and 7 more figures

Theorems & Definitions (3)

  • Remark 3.1
  • Remark 3.2
  • Remark 3.3