Transformations in Learned Image Compression from a Modulation Perspective

TL;DR

This work addresses improving learned image compression by redesigning transformation modules from a modulation perspective. It treats LIC as a communication system, introduces Transform based on Signal Modulation (TSM), and derives nonlinear variants TPM, TFM, and TJM, complemented by a residual block (ResTSM). Empirical results show BD-rate reductions over GDN baselines (e.g., 3.52% on Kodak with a hyperprior/context model) and competitive performance with lower complexity, validating the cross-disciplinary approach. The study demonstrates that communication-theory guidance can yield practical gains in LIC without substantial architectural overhead, enhancing robustness across datasets and backbones.

Abstract

In this paper, a unified transformation method in learned image compression(LIC) is proposed from the perspective of modulation. Firstly, the quantization in LIC is considered as a generalized channel with additive uniform noise. Moreover, the LIC is interpreted as a particular communication system according to the consistency in structures and optimization objectives. Thus, the technology of communication systems can be applied to guide the design of modules in LIC. Furthermore, a unified transform method based on signal modulation (TSM) is defined. In the view of TSM, the existing transformation methods are mathematically reduced to a linear modulation. A series of transformation methods, e.g. TPM and TJM, are obtained by extending to nonlinear modulation. The experimental results on various datasets and backbone architectures verify that the effectiveness and robustness of the proposed method. More importantly, it further confirms the feasibility of guiding LIC design from a communication perspective. For example, when backbone architecture is hyperprior combining context model, our method achieves 3.52$\%$ BD-rate reduction over GDN on Kodak dataset without increasing complexity.

Figures (8)

• Figure 1: The transformations in learned image compression are considered as the modulation of the signal. Existing methods can be classified as linear modulation. By extending to nonlinear modulation, a series of transformation methods are yielded.
• Figure 2: Illustration of the consistency between communication system and learned image compression. The top row represents communication system, where $S, Z, \hat{Z},\hat{S}$ denote the original signal, the transmitted signal, the received signal and the estimate signal respectively, and $E$ represents the distortion between $S$ and $\hat{S}$. $I$ represents the mutual information between $Z$ and $\hat{Z}$. The bottom row represents communication system, where $s, z, \hat{z},\hat{s}$ denote the original image, the latent vertor, the discrete reconstructed vector and the restructured image respectively, and $D$ stands for the distortion between $s$ and $\hat{s}$. $R$ stands for the bitrate of $\hat{z}$.
• Figure 3: Illustration of the proposed transforms based on Signal Modlation (TSM) and its residual block (ResTSM). The switch means that three branches $A,\ \phi \ \text{and} \ \omega$ can be controlled to obtain different transformation methods: when only $A$ or $\phi$ or $\omega$ branch is available, this corresponds to TAM, TPM and TFM respectively.
• Figure 4: Illustration of our methods on ICLR2018balle2018variational network architecture. MU is a collective term for our methods, including TPM, TFM, TJM and its residual block (ResTSM). Entropy model represents the hyperprior model. Q represents quantization, and AE, AD represent arithmetic encoder and arithmetic decoder, respectively. Convolution parameters are denoted as: number of filters $\times$ kernel support height $\times$ kernel support width / down- or upsampling stride, where ↑ indicates upsampling and ↓ downsampling.
• Figure 5: Rate-Distortion Performance of our proposed transformation and the other methods on the Kodak and Tecnick datasets. The basic model is CVPR2021Cui_2021_CVPR, and all experiments only operate on the nonlinear transformation without changing the whole neural network architecture and hyperparameters. PSNR and MS-SSIM are used as the distortion metrics. The MS-SSIM values are converted into decibels $-10log_{10}(1-d)$ where $d$ refers to the MS-SSIM value, for a clear illustration. The anchor for calculating the BD-rate gain is CVPR2021Cui_2021_CVPR with GDN while distortion is measured by PSNR.