HybridFlow: Infusing Continuity into Masked Codebook for Extreme Low-Bitrate Image Compression

TL;DR

This work addresses the challenge of ultra-low-bitrate image compression with learned methods by proposing HybridFlow, a dual-stream LIC framework that fuses a discrete codebook stream with a continuous feature stream. A masked-token predictor, guided by continuous-domain information through cross-attention, reconstructs full codeword indices from partial disclosures, while a bridging correction network fuses both streams at the pixel level to maintain fidelity and perceptual quality. The method achieves strong PSNR and LPIPS performance at rates below $0.05\,\mathrm{bpp}$, outperforming single-stream codebook or continuous LIC methods and offering a practical approach for extreme compression scenarios. Additional gains come from complexity-aware dynamic masking, which allocates bits adaptively to high-detail regions, improving efficiency without sacrificing quality.

Abstract

This paper investigates the challenging problem of learned image compression (LIC) with extreme low bitrates. Previous LIC methods based on transmitting quantized continuous features often yield blurry and noisy reconstruction due to the severe quantization loss. While previous LIC methods based on learned codebooks that discretize visual space usually give poor-fidelity reconstruction due to the insufficient representation power of limited codewords in capturing faithful details. We propose a novel dual-stream framework, HyrbidFlow, which combines the continuous-feature-based and codebook-based streams to achieve both high perceptual quality and high fidelity under extreme low bitrates. The codebook-based stream benefits from the high-quality learned codebook priors to provide high quality and clarity in reconstructed images. The continuous feature stream targets at maintaining fidelity details. To achieve the ultra low bitrate, a masked token-based transformer is further proposed, where we only transmit a masked portion of codeword indices and recover the missing indices through token generation guided by information from the continuous feature stream. We also develop a bridging correction network to merge the two streams in pixel decoding for final image reconstruction, where the continuous stream features rectify biases of the codebook-based pixel decoder to impose reconstructed fidelity details. Experimental results demonstrate superior performance across several datasets under extremely low bitrates, compared with existing single-stream codebook-based or continuous-feature-based LIC methods.

Figures (8)

• Figure 1: The dual-stream "HybridFlow" framework for extreme low-bitrate image compression.
• Figure 2: Training pipeline for our proposed framework.
• Figure 3: Candidate mask schedules for codeword indices map (shown in the 8x8 indices map). The average mask ratio for each schedule is 50%, 75%, 90.3%, 93.75%, respectively.
• Figure 4: Effectiveness of our mask predictor. The first to last row separately have a mask schedule of 1_4, 1_9, 1_16.
• Figure 5: Effectiveness of continuity-assisted pixel decoder. The red boxes emphasize specific regions where the duplicate decoder leverages continuous-domain information to effectively correct codebook-based deviation.