Multiscale Augmented Normalizing Flows for Image Compression

TL;DR

The paper tackles the limited efficiency of high-quality image compression in non-invertible learned pipelines by employing invertible latent variable models with a hierarchical, multiscale latent space. It introduces two architectures, M-ANFIC and MS-ANFIC, which extend augmented normalizing flows (ANFIC) with scalable latent representations, including an explicit latent split network for the final layer. Experimental results on TECNICK and training on Vimeo-90k demonstrate BD-rate reductions of up to about 7–9% over single-scale baselines, with MS-ANFIC achieving the strongest gains and reduced parameter counts. The work shows that architectural choices, particularly multiscale latent representations and invertibility, can substantially improve learning-based image compression and enable adaptive bit allocation across image regions.

Abstract

Most learning-based image compression methods lack efficiency for high image quality due to their non-invertible design. The decoding function of the frequently applied compressive autoencoder architecture is only an approximated inverse of the encoding transform. This issue can be resolved by using invertible latent variable models, which allow a perfect reconstruction if no quantization is performed. Furthermore, many traditional image and video coders apply dynamic block partitioning to vary the compression of certain image regions depending on their content. Inspired by this approach, hierarchical latent spaces have been applied to learning-based compression networks. In this paper, we present a novel concept, which adapts the hierarchical latent space for augmented normalizing flows, an invertible latent variable model. Our best performing model achieved average rate savings of more than 7% over comparable single-scale models.

Figures (7)

• Figure 1: Architecture overview of M-ANFIC. Conv$c/k/s\downarrow$ denotes a convolution layer with $c$ output channels, a kernel size of $k\times k$ and a downsampling factor of $s$. TConv$c/k/s\uparrow$ denotes an identical parametrized transposed convolution. Filled circles after a convolution represent GDNs/IGDNs Balle2016. Shaded area marks the novel components.
• Figure 2: Structure of LSUnit (Type A). Notation analogous to Fig. \ref{['fig:m_anfic_architecture']}. Two signals entering the same convolution corresponds to a concatenation along the channel dimension. Latent space is masked according to the external mask.
• Figure 3: Structure of LSUnit (Type B). Notation analogous to Figs. \ref{['fig:m_anfic_architecture']} and \ref{['fig:lsunit_A']}. Blank rhombuses denote leaky ReLUs. The architecture of the conditional hyperprior is identical to the hyperprior of ANFIC Ho2021b but with $\bm{v}_{n+1}$ concatenated to the input of the first encoding and last decoding layer.
• Figure 4: Architecture overview of MS-ANFIC. Notation analogous to Fig. \ref{['fig:m_anfic_architecture']}. Shaded area marks the novel components.
• Figure 5: Latent split network. Notation analogous to Fig. \ref{['fig:m_anfic_architecture']}.