Joint Multi-scale Gated Transformer and Prior-guided Convolutional Network for Learned Image Compression

TL;DR

The paper addresses the RD performance gap in learned image compression by introducing two novel components: prior-guided convolution (PGConv) to strengthen local feature extraction, and a multi-scale gated transformer (MGT) to enhance non-local context modeling. These are integrated into a joint framework called MGTPCN, combining an analysis/synthesis transform pair with an entropy model. Empirical results on Kodak and Tecnick show that MGTPCN achieves superior rate-distortion performance with favorable parameter and computation trade-offs, outperforming several state-of-the-art methods. The work demonstrates that targeted architectural innovations in both local and non-local transforms can yield meaningful gains in learned image compression efficiency and reconstruction fidelity.

Abstract

Recently, learned image compression methods have made remarkable achievements, some of which have outperformed the traditional image codec VVC. The advantages of learned image compression methods over traditional image codecs can be largely attributed to their powerful nonlinear transform coding. Convolutional layers and shifted window transformer (Swin-T) blocks are the basic units of neural networks, and their representation capabilities play an important role in nonlinear transform coding. In this paper, to improve the ability of the vanilla convolution to extract local features, we propose a novel prior-guided convolution (PGConv), where asymmetric convolutions (AConvs) and difference convolutions (DConvs) are introduced to strengthen skeleton elements and extract high-frequency information, respectively. A re-parameterization strategy is also used to reduce the computational complexity of PGConv. Moreover, to improve the ability of the Swin-T block to extract non-local features, we propose a novel multi-scale gated transformer (MGT), where dilated window-based multi-head self-attention blocks with different dilation rates and depth-wise convolution layers with different kernel sizes are used to extract multi-scale features, and a gate mechanism is introduced to enhance non-linearity. Finally, we propose a novel joint Multi-scale Gated Transformer and Prior-guided Convolutional Network (MGTPCN) for learned image compression. Experimental results show that our MGTPCN surpasses state-of-the-art algorithms with a better trade-off between performance and complexity.

Figures (5)

• Figure 1: The architecture of the proposed MGTPCN. $Q$ denotes quantization. AE and AD denote the arithmetic encoding and arithmetic decoding, respectively.
• Figure 2: DW-MSAs with different dilation rates.
• Figure 3: RD curves of different methods on the Kodak dataset.
• Figure 4: RD curves of different methods on the Tecnick dataset.
• Figure 5: Visualization of the decoded image from the Kodak dataset.