Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression

Shima Rafiei, Zahra Nabizadeh Shahr Babak, Shadrokh Samavi, Shahram Shirani

TL;DR

RAVQ-HoloNet tackles the data-telemetry bottleneck in holographic AR/VR by introducing a rate-adaptive, vector-quantized compression framework that unifies complex-valued hologram encoding with phase-only reconstruction. The method leverages a hierarchical VQ latent space with EMA codebooks and a Seq2seq bitrate tuner to enable continuous bitrate control within a single model, avoiding multiple-model maintenance. A two-stage training regime couples a physics-informed ASM-based reconstruction loss with traditional VQ losses, yielding competitive rate–distortion performance; on 2D holograms, it achieves up to -33.91% BD-Rate improvement and $1.02$ dB BD-PSNR over state-of-the-art DPRC at ultra-low bitrates. The approach shows clear practical impact for bandwidth- and storage-constrained devices, and extends naturally to RGB-D and 3D holographic representations, as well as gaze-driven compression strategies for perceptual prioritization.

Abstract

Holography offers significant potential for AR/VR applications, yet its adoption is limited by the high demands of data compression. Existing deep learning approaches generally lack rate adaptivity within a single network. We present RAVQ-HoloNet, a rate-adaptive vector quantization framework that achieves high-fidelity reconstructions at low and ultra-low bit rates, outperforming current state-of-the-art methods. In low bit, our method exceeds by -33.91% in BD-Rate and achieves a BD-PSNR of 1.02 dB from the best existing method demonstrated by the rate-distortion curve.

RAVQ-HoloNet: Rate-Adaptive Vector-Quantized Hologram Compression

TL;DR

RAVQ-HoloNet tackles the data-telemetry bottleneck in holographic AR/VR by introducing a rate-adaptive, vector-quantized compression framework that unifies complex-valued hologram encoding with phase-only reconstruction. The method leverages a hierarchical VQ latent space with EMA codebooks and a Seq2seq bitrate tuner to enable continuous bitrate control within a single model, avoiding multiple-model maintenance. A two-stage training regime couples a physics-informed ASM-based reconstruction loss with traditional VQ losses, yielding competitive rate–distortion performance; on 2D holograms, it achieves up to -33.91% BD-Rate improvement and dB BD-PSNR over state-of-the-art DPRC at ultra-low bitrates. The approach shows clear practical impact for bandwidth- and storage-constrained devices, and extends naturally to RGB-D and 3D holographic representations, as well as gaze-driven compression strategies for perceptual prioritization.

Abstract

Holography offers significant potential for AR/VR applications, yet its adoption is limited by the high demands of data compression. Existing deep learning approaches generally lack rate adaptivity within a single network. We present RAVQ-HoloNet, a rate-adaptive vector quantization framework that achieves high-fidelity reconstructions at low and ultra-low bit rates, outperforming current state-of-the-art methods. In low bit, our method exceeds by -33.91% in BD-Rate and achieves a BD-PSNR of 1.02 dB from the best existing method demonstrated by the rate-distortion curve.

Paper Structure

This paper contains 21 sections, 9 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Hologram Computation
  3. Hologram Compression
  4. Proposed Method
  5. Complex Hologram Generation
  6. Hologram Compression
  7. Rate-Adaptive Seq2seq Model
  8. Communication
  9. Training and Loss Functions
  10. Experimental Results
  11. Dataset
  12. Development Platform
  13. Previous Work
  14. Compression Viewpoint
  15. 1) Reducing the number of code vectors in the dictionary.
  16. ...and 6 more sections

Figures (7)

  • Figure 1: The block diagram of the proposed method. Illustration of the codebook adaptation framework. The process begins with the original codebook $\mathbf{e} = \{e_1, e_2, \ldots, e_K\}$. These are passed through a Seq2seq model composed of LSTM units to produce an intermediate representation. A subsequent decoder transforms this representation into the adapted codebook $\tilde{\mathbf{e}} = \{\tilde{e}_1, \tilde{e}_2, \ldots, \tilde{e}_K\}$, illustrated for the top and bottom codebooks.
  • Figure 2: A simplified illustration of the sender-receiver hologram communication framework.
  • Figure 3: A quantitative result of VQ-HoloNet vs. DPRC
  • Figure 4: Probability of codebook indices selected in each layer of the Bottom and Top across the test set, for each channel.
  • Figure 5: (a, b, c) Rate-distortion performance indicated as quality metrics vs. Bpp, (d) Code vector utilization
  • ...and 2 more figures