MLICv2: Enhanced Multi-Reference Entropy Modeling for Learned Image Compression

TL;DR

This paper proposes MLICv2 and MLICv2 , enhanced successors that systematically address limitations through improved transform design, advanced entropy modeling, and exploration of the potential of instance-specific optimization.

Abstract

Recent advances in learned image compression (LIC) have achieved remarkable performance improvements over traditional codecs. Notably, the MLIC series-LICs equipped with multi-reference entropy models-have substantially surpassed conventional image codecs such as Versatile Video Coding (VVC) Intra. However, existing MLIC variants suffer from several limitations: performance degradation at high bitrates due to insufficient transform capacity, suboptimal entropy modeling that fails to capture global correlations in initial slices, and lack of adaptive channel importance modeling. In this paper, we propose MLICv2 and MLICv2+, enhanced successors that systematically address these limitations through improved transform design, dvanced entropy modeling, and exploration of the potential of instance-specific optimization. For transform enhancement, we introduce a lightweight token mixing block inspired by the MetaFormer architecture, which effectively mitigates high-bitrate performance degradation while maintaining computational efficiency. For entropy modeling improvements, we propose hyperprior-guided global correlation prediction to extract global context even in the initial slice of latent representation, complemented by a channel reweighting module that dynamically emphasizes informative channels. We further explore enhanced positional embedding and guided selective compression strategies for superior context modeling. Additionally, we apply the Stochastic Gumbel Annealing (SGA) to demonstrate the potential for further performance improvements through input-specific optimization. Extensive experiments demonstrate that MLICv2 and MLICv2+ achieve state-of-the-art results, reducing Bjøntegaard-Delta Rate by 16.54%, 21.61%, 16.05% and 20.46%, 24.35%, 19.14% on Kodak, Tecnick, and CLIC Pro Val datasets, respectively, compared to VTM-17.0 Intra.

Figures (14)

• Figure 1: BD-Rate and decoding time during inference on CLIC Professional Valid CLIC2020 with 2K resolution. The bubble size indicates the number of model parameters.
• Figure 2: Model evolution of the MLIC series.
• Figure 3: The overall architecture of MLICv2/MLICv2$^+$. $g_a$ is the analysis transform, $g_s$ is the synthesis transform, $h_a$ is the hyper analysis, and $h_s$ is the hyper synthesis. $g_{ep}$ is the entropy parameter module. $f_s$ is the selective compression predictive module. $g_{lrp}$ is the latent residual prediction module minnen2020channel. AE and AD are arithmetic encoding and decoding. $\uparrow$ denotes upsampling and $\downarrow$ denotes downsampling. ${\boldsymbol{x}}$ is the input image and $\hat{\boldsymbol x}$ is the reconstructed image. $\boldsymbol{y}$ is the latent representation and $\hat{\boldsymbol{y}}$ is the quantized latent representation. $\hat{\boldsymbol y}^i$ is the $i$-th slice of $\hat{\boldsymbol{y}}$. $\boldsymbol{\mu}^i, \boldsymbol{\sigma}^i$ are the mean and scale of $\hat{\boldsymbol{y}}^i$. $\boldsymbol{s}^i$ is the predicted selective coding map. $\boldsymbol{r}^i$ is the predicted residual. $\hat{\boldsymbol{z}}$ is the side information. $M, N$ are the channel numbers, which are $320, 192$ in MLICv2/MLICv2$^+$. "Refine" is employed in MLICv2$^+$.
• Figure 4: Visualization of the channels of hyperprior $\hat{\boldsymbol{H}}$ ($\lambda=0.013$). $\hat{\boldsymbol{H}}$ extracted from the latent representation of Image Kodim20 exhibits similar global similarity to original Kodim20.
• Figure 5: Proposed context reweighting module.

Theorems & Definitions (1)

• Definition 4.1: Mean Attention Weight