HVQ-CGIC: Enabling Hyperprior Entropy Modeling for VQ-Based Controllable Generative Image Compression

TL;DR

HVQ-CGIC tackles the bottleneck in VQ-based generative image compression: non-adaptive entropy modeling of discrete VQ indices. By predicting Gaussian parameters in embedding space and converting them to content-adaptive index probabilities via Mahalanobis-distance softmax, the method enables end-to-end rate-distortion optimization for VQ indices. The architecture uses three granularity levels with independent hyperpriors and a probability-adaptive decoder, achieving state-of-the-art RD-perception trade-offs and substantial bitrate reductions on Kodak, while maintaining practical encoding speeds. This work lays a foundation for RD-controlled, VQGAN-based compression by extending hyperprior entropy modeling to discrete indices and integrates it into a lightweight, scalable framework.

Abstract

Generative learned image compression methods using Vector Quantization (VQ) have recently shown impressive potential in balancing distortion and perceptual quality. However, these methods typically estimate the entropy of VQ indices using a static, global probability distribution, which fails to adapt to the specific content of each image. This non-adaptive approach leads to untapped bitrate potential and challenges in achieving flexible rate control. To address this challenge, we introduce a Controllable Generative Image Compression framework based on a VQ Hyperprior, termed HVQ-CGIC. HVQ-CGIC rigorously derives the mathematical foundation for introducing a hyperprior to the VQ indices entropy model. Based on this foundation, through novel loss design, to our knowledge, this framework is the first to introduce RD balance and control into vector quantization-based Generative Image Compression. Cooperating with a lightweight hyper-prior estimation network, HVQ-CGIC achieves a significant advantage in rate-distortion (RD) performance compared to current state-of-the-art (SOTA) generative compression methods. On the Kodak dataset, we achieve the same LPIPS as Control-GIC, CDC and HiFiC with an average of 61.3% fewer bits. We posit that HVQ-CGIC has the potential to become a foundational component for VQGAN-based image compression, analogous to the integral role of the HyperPrior framework in neural image compression.

Figures (7)

• Figure 1: Our HVQ-CGIC breaks through the limitation that VQ cannot design hyperprior entropy models, and achieves significantly lower bitrates than li2024once on Kodak under the same downsampling factor.
• Figure 2: HVQ-CGIC flowchart, details see \ref{['sec:method_design']}. $\Theta$: represents element-wise multiplication here.
• Figure 3: Scalar quantization hyperprior model (a) and our proposed vector quantization hyperprior models (b)(c). Detailed introduction is in §\ref{['sec:Math']}.
• Figure 4: Comparison of Methods on Kodak Dataset.
• Figure 5: Conduct quantitative comparisons with the listed Baseline Methods.