Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ResiTok: A Resilient Tokenization-Enabled Framework for Ultra-Low-Rate and Robust Image Transmission

Zhenyu Liu, Yi Ma, Rahim Tafazolli

TL;DR

ResiTok introduces a resilient 1D tokenization framework for ultra-low-rate image transmission that progressively degrades visual quality through hierarchical key and detail tokens. A novel zero-out training procedure, combined with channel-adaptive coding and modulation, enables robust reconstruction from incomplete token sets under severe bandwidth constraints. The approach demonstrates superior semantic fidelity and visual quality over state-of-the-art methods across challenging datasets and channel conditions, while maintaining a practical computational footprint. The work offers a scalable solution for real-time image delivery in bandwidth-limited wireless networks, with broad applicability to semantic communications and resilient remote sensing.

Abstract

Real-time transmission of visual data over wireless networks remains highly challenging, even when leveraging advanced deep neural networks, particularly under severe channel conditions such as limited bandwidth and weak connectivity. In this paper, we propose a novel Resilient Tokenization-Enabled (ResiTok) framework designed for ultra-low-rate image transmission that achieves exceptional robustness while maintaining high reconstruction quality. By reorganizing visual information into hierarchical token groups consisting of essential key tokens and supplementary detail tokens, ResiTok enables progressive encoding and graceful degradation of visual quality under constrained channel conditions. A key contribution is our resilient 1D tokenization method integrated with a specialized zero-out training strategy, which systematically simulates token loss during training, empowering the neural network to effectively compress and reconstruct images from incomplete token sets. Furthermore, the channel-adaptive coding and modulation design dynamically allocates coding resources according to prevailing channel conditions, yielding superior semantic fidelity and structural consistency even at extremely low channel bandwidth ratios. Evaluation results demonstrate that ResiTok outperforms state-of-the-art methods in both semantic similarity and visual quality, with significant advantages under challenging channel conditions.

ResiTok: A Resilient Tokenization-Enabled Framework for Ultra-Low-Rate and Robust Image Transmission

TL;DR

ResiTok introduces a resilient 1D tokenization framework for ultra-low-rate image transmission that progressively degrades visual quality through hierarchical key and detail tokens. A novel zero-out training procedure, combined with channel-adaptive coding and modulation, enables robust reconstruction from incomplete token sets under severe bandwidth constraints. The approach demonstrates superior semantic fidelity and visual quality over state-of-the-art methods across challenging datasets and channel conditions, while maintaining a practical computational footprint. The work offers a scalable solution for real-time image delivery in bandwidth-limited wireless networks, with broad applicability to semantic communications and resilient remote sensing.

Abstract

Real-time transmission of visual data over wireless networks remains highly challenging, even when leveraging advanced deep neural networks, particularly under severe channel conditions such as limited bandwidth and weak connectivity. In this paper, we propose a novel Resilient Tokenization-Enabled (ResiTok) framework designed for ultra-low-rate image transmission that achieves exceptional robustness while maintaining high reconstruction quality. By reorganizing visual information into hierarchical token groups consisting of essential key tokens and supplementary detail tokens, ResiTok enables progressive encoding and graceful degradation of visual quality under constrained channel conditions. A key contribution is our resilient 1D tokenization method integrated with a specialized zero-out training strategy, which systematically simulates token loss during training, empowering the neural network to effectively compress and reconstruct images from incomplete token sets. Furthermore, the channel-adaptive coding and modulation design dynamically allocates coding resources according to prevailing channel conditions, yielding superior semantic fidelity and structural consistency even at extremely low channel bandwidth ratios. Evaluation results demonstrate that ResiTok outperforms state-of-the-art methods in both semantic similarity and visual quality, with significant advantages under challenging channel conditions.
Paper Structure (22 sections, 10 equations, 5 figures, 1 table)

This paper contains 22 sections, 10 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Resilient Tokenization Communication Framework
  3. Zero-Out Training Procedure
  4. Tokenization Architecture
  5. Progressive Zero-Out Training Strategy
  6. Resilient Tokenization Enabled Ultra-Low-Rate and Robust Image Transmission
  7. Tokenization Process
  8. Resilience-Aware Coding and Modulation
  9. Progressive Source Coding
  10. Channel-Aware Coding and Modulation
  11. Reception and Reconstruction
  12. Demodulation and Channel Decoding
  13. Token Reconstruction with Zero Padding
  14. Image Reconstruction
  15. Performance Evaluations
  16. ...and 7 more sections

Figures (5)

  • Figure 1: The proposed resilient tokenization communication framework for ultra-low-rate and robust image transmission.
  • Figure 2: Training framework for progressive tokenization.
  • Figure 3: Visual quality and CBR comparison of the proposed approach against existing methods when SNR = 6dB using a 16-QAM modulation.
  • Figure 4: Performance comparison versus CBR when SNR = 6dB: (a) CLIP score on CLIC2021 dataset, (b) PSNR on CLIC2021 dataset, (c) CLIP score on Kodak dataset, (d) PSNR on Kodak dataset.
  • Figure 5: Performance versus SNR over Kodak dataset when CBR = $\frac{1}{256}$: (a) CLIP Score and (b) PSNR metrics.