Sensitivity Decouple Learning for Image Compression Artifacts Reduction

TL;DR

The paper tackles JPEG and similar compression artifacts that degrade both visual quality and downstream vision tasks. It introduces sensitivity decouple learning to separate intrinsic compression attributes into compression-insensitive features for high-level semantics and compression-sensitive features for compression degree, implemented via two auto-encoders and a Dual Awareness Guidance Network (DAGN). DAGN employs a compression-insensitive guidance module, a compression-sensitive guidance module, and a cross-feature fusion module to guide decoding, achieving state-of-the-art PSNR gains (e.g., ~$2.06$ dB on BSD500) and improved performance on object detection and semantic segmentation after artifact reduction. The method demonstrates strong empirical results across standard benchmarks (LIVE1, BSD500, LIU4K) and real-world images, with comparable efficiency to non-transformer baselines and superior task performance, illustrating practical impact for both image restoration and computer vision pipelines.

Abstract

With the benefit of deep learning techniques, recent researches have made significant progress in image compression artifacts reduction. Despite their improved performances, prevailing methods only focus on learning a mapping from the compressed image to the original one but ignore the intrinsic attributes of the given compressed images, which greatly harms the performance of downstream parsing tasks. Different from these methods, we propose to decouple the intrinsic attributes into two complementary features for artifacts reduction,ie, the compression-insensitive features to regularize the high-level semantic representations during training and the compression-sensitive features to be aware of the compression degree. To achieve this, we first employ adversarial training to regularize the compressed and original encoded features for retaining high-level semantics, and we then develop the compression quality-aware feature encoder for compression-sensitive features. Based on these dual complementary features, we propose a Dual Awareness Guidance Network (DAGN) to utilize these awareness features as transformation guidance during the decoding phase. In our proposed DAGN, we develop a cross-feature fusion module to maintain the consistency of compression-insensitive features by fusing compression-insensitive features into the artifacts reduction baseline. Our method achieves an average 2.06 dB PSNR gains on BSD500, outperforming state-of-the-art methods, and only requires 29.7 ms to process one image on BSD500. Besides, the experimental results on LIVE1 and LIU4K also demonstrate the efficiency, effectiveness, and superiority of the proposed method in terms of quantitative metrics, visual quality, and downstream machine vision tasks.

Figures (13)

• Figure 1: The motivation of the proposed sensitivity decouple learning, which explicitly mines the intrinsic attributes from the given compressed image. The proposed method decouples the intrinsic image attributes into compression-insensitive features for high-level semantics ( e.g., hat and face) and compression-sensitive features for low-level cues ( e.g., edge and shape).
• Figure 2: An original image and its JPEG compressed version with $\mathrm{QF}=10$ that shows the relationship between the intrinsic attributes and compression. Some intrinsic attributes are changed by JPEG compression, such as dimples, while others are maintained, such as sphere.
• Figure 3: The training of compression-insensitive auto-encoder. The discriminator takes the features learned by the compression-insensitive encoder as the input and generates the probability that the features are from original images. By training the auto-encoder to fool the differentiable discriminator network, we obtain the compression-insensitive encoder that could extract compression-insensitive features from images.
• Figure 4: Visualization of the compression-insensitive features learned from womanhat in LIVE1 and 206062 in BSD500.
• Figure 5: The training of compression-sensitive auto-encoder. The QF predictor takes the features learned by the compression-sensitive encoder as the input and generates the QF distribution. By training the auto-encoder with the goal of recognizing the QF, we obtain a compression-sensitive encoder that could extract compression-insensitive features from images.