All-in-One Image Compression and Restoration

TL;DR

This work tackles practical image compression under diverse degradations by proposing a unified all-in-one framework that jointly restores and compresses images. It introduces content information aggregation and degradation representation aggregation through a Hybrid-Attention Transformer Block (HATB) that combines Channel-wise Group Attention ($\text{C-GA}$) and Spatially Decoupled Attention ($\text{S-DA}$) within an encoder–decoder–entropy model, enabling the system to preserve structure while suppressing degradations without degradation priors. The model achieves superior rate-distortion performance on degraded inputs while maintaining or closely matching performance on clean images, with strong generalization to real-world degradations and improved efficiency over cascaded or naive joint methods. The approach is validated with extensive experiments across weather- and noise-related degradations and downstream tasks, and is supported by a public code release.

Abstract

Visual images corrupted by various types and levels of degradations are commonly encountered in practical image compression. However, most existing image compression methods are tailored for clean images, therefore struggling to achieve satisfying results on these images. Joint compression and restoration methods typically focus on a single type of degradation and fail to address a variety of degradations in practice. To this end, we propose a unified framework for all-in-one image compression and restoration, which incorporates the image restoration capability against various degradations into the process of image compression. The key challenges involve distinguishing authentic image content from degradations, and flexibly eliminating various degradations without prior knowledge. Specifically, the proposed framework approaches these challenges from two perspectives: i.e., content information aggregation, and degradation representation aggregation. Extensive experiments demonstrate the following merits of our model: 1) superior rate-distortion (RD) performance on various degraded inputs while preserving the performance on clean data; 2) strong generalization ability to real-world and unseen scenarios; 3) higher computing efficiency over compared methods. Our code is available at https://github.com/ZeldaM1/All-in-one.

Figures (21)

• Figure 1: Results of typical solutions for degraded image compression, where BPP/PSNR/MS-SSIM are reported for each method. The image codec EVC (designed for clean images) allocates extra bits to preserve degradations. Cascaded solutions (e.g., Restormer + EVC ) amplify artifacts introduced in the restoration stage.
• Figure 2: The proposed all-in-one framework, which consists of a feature encoder $\mathcal{G}{\phi_a}$, a feature decoder $\mathcal{G}{\phi_s}$ and a spatial entropy model. The HATB effectively models long-range dependencies with the C-GA, and captures discriminative representations with the S-DA.
• Figure 3: Visualization of the input feature $\mathbf{X}^{CGA}$ and output feature $\hat{\mathbf{X}}^{CGA}$ in C-GA. Although degradations and image signals are closely intertwined in the input features, the C-GA effectively separates degradations from the image content (e.g., the elephant is distinguished from the rain streaks in the yellow box), thereby preserving image signals.
• Figure 4: Visual comparisons of output feature $\hat{\mathbf{X}}^{SDA}$ in S-DA and t-SNE results, where SD indicates spatial decoupling. As can be seen, the design of spatial decoupling helps to effectively extract discriminative degradation representations (e.g., the snow spots in the yellow box and distinct clusters in the t-SNE map).
• Figure 5: RD performance evaluation on the RESIDE reside, CSD csd and Rain1400 rain1400 dataset, where we evaluate the results with both PSNR and MS-SSIM.