Joint Hierarchical Priors and Adaptive Spatial Resolution for Efficient Neural Image Compression

TL;DR

The paper tackles efficient neural image compression by combining Transformer-based nonlinear transform coding with a channel-wise autoregressive entropy model and an adaptive spatial resolution module. It introduces ICT (Transformer-based channel-wise autoregressive prior) and AICT (scale-adaptation module built on a ConvNeXt pre-/post-processor) to capture global/local latent context and adapt resolution, achieving favorable rate–distortion performance and lower decoding latency. Key contributions include a spatio-channel entropy model with a tiny Transformer slice context, a Swin Transformer-based analysis/synthesis transform, and a scale-adaptation pipeline validated by extensive experiments showing BD-rate reductions on standard benchmarks. The approach demonstrates strong potential for hardware-friendly neural codecs, enabling real-time or near-real-time image compression with competitive performance relative to VVC and prior neural methods.

Abstract

Recently, the performance of neural image compression (NIC) has steadily improved thanks to the last line of study, reaching or outperforming state-of-the-art conventional codecs. Despite significant progress, current NIC methods still rely on ConvNet-based entropy coding, limited in modeling long-range dependencies due to their local connectivity and the increasing number of architectural biases and priors, resulting in complex underperforming models with high decoding latency. Motivated by the efficiency investigation of the Tranformer-based transform coding framework, namely SwinT-ChARM, we propose to enhance the latter, as first, with a more straightforward yet effective Tranformer-based channel-wise auto-regressive prior model, resulting in an absolute image compression transformer (ICT). Through the proposed ICT, we can capture both global and local contexts from the latent representations and better parameterize the distribution of the quantized latents. Further, we leverage a learnable scaling module with a sandwich ConvNeXt-based pre-/post-processor to accurately extract more compact latent codes while reconstructing higher-quality images. Extensive experimental results on benchmark datasets showed that the proposed framework significantly improves the trade-off between coding efficiency and decoder complexity over the versatile video coding (VVC) reference encoder (VTM-18.0) and the neural codec SwinT-ChARM. Moreover, we provide model scaling studies to verify the computational efficiency of our approach and conduct several objective and subjective analyses to bring to the fore the performance gap between the adaptive image compression transformer (AICT) and the neural codec SwinT-ChARM.

Figures (12)

• Figure 1: A high-level diagram of the proposed AICT solution. ChARM refers to the Transformer-based channel-wise autoregressive prior model, and $s$ represents the resizing parameter predicted by the neural estimator $(s \in \mathbb{R} \cap [0, 1])$.
• Figure 2: Overall AICT Framework. We illustrate the image compression diagram of our AICT with hyperprior and Swin Transformer based charm, and scale adaptation module. The rpn, ConvNeXt block, and Swin Transformer block architectures are respectively detailed in (a), (c), and (d) Fig. \ref{['blocks']}.
• Figure 3: Detailed description of block architectures: (a) rpn, (b) ResBlock, (c) ConvNeXt Block, and (d) Swin Transformer Block. DConv2D(.) stands for depthwise 2D convolution, LayerNorm for the layer normalization, Dense(.) for the densely-connected neural network layer, and gelu for the activation.
• Figure 4: Spatio-channel entropy coding. $g_a$, $h_s$, $g^{i}_{cm}$, AE, and AD stand for analysis and hyper-synthesis transforms, the $i^{th}$ context model, and arithmetic encoder/decoder, respectively. $\{y^1,\dots,y^s\}$ stands for the already-decoded latent slices, where $\{s \in \mathbb{N} \mid 1 \leq s \leq 5 \}$ is the number of supported slices.
• Figure 5: Number of images per dataset per pixel count in megapixel (Mpx).