OAPT: Offset-Aware Partition Transformer for Double JPEG Artifacts Removal

TL;DR

Double JPEG artifacts are exacerbated by non-aligned block shifts, making restoration challenging for standard single-pass models. OAPT addresses this with a compression offset predictor and Hybrid Partition Attention Blocks that cluster pattern-specific features and perform pattern-aware attention, enabling robust recovery across aligned and non-aligned double JPEG scenarios. Empirical results show about 0.16 dB PSNR improvement over the state of the art on grayscale double JPEG restoration, with additional robustness to real-world JPEGs and the ability to boost other transformer-based methods via a lightweight pattern clustering plugin. The approach provides a practical, plug-in capable framework for non-aligned double JPEG restoration and highlights a path toward more generalizable, pattern-aware image restoration.

Abstract

Deep learning-based methods have shown remarkable performance in single JPEG artifacts removal task. However, existing methods tend to degrade on double JPEG images, which are prevalent in real-world scenarios. To address this issue, we propose Offset-Aware Partition Transformer for double JPEG artifacts removal, termed as OAPT. We conduct an analysis of double JPEG compression that results in up to four patterns within each 8x8 block and design our model to cluster the similar patterns to remedy the difficulty of restoration. Our OAPT consists of two components: compression offset predictor and image reconstructor. Specifically, the predictor estimates pixel offsets between the first and second compression, which are then utilized to divide different patterns. The reconstructor is mainly based on several Hybrid Partition Attention Blocks (HPAB), combining vanilla window-based self-attention and sparse attention for clustered pattern features. Extensive experiments demonstrate that OAPT outperforms the state-of-the-art method by more than 0.16dB in double JPEG image restoration task. Moreover, without increasing any computation cost, the pattern clustering module in HPAB can serve as a plugin to enhance other transformer-based image restoration methods. The code will be available at https://github.com/QMoQ/OAPT.git .

Figures (9)

• Figure 1: The demonstration of double compression.
• Figure 2: The architecture of Offset-Aware Partition Transformer(OAPT). In PC-MSA, P-cluster represents Pattern clustering and Inv P-cluster stands for the inverse operation of pattern clustering. (S) means equipping with window shifting operation.
• Figure 3: The demonstration of pattern clustering module.
• Figure 4: Visual comparison for various methods on benchmark datasets. The compression type is $\text{QF}_1=30$, $\text{QF}_2=50$, offsets=$(4, 4)$.
• Figure 5: NIQE$\downarrow$ / BRISQUE$\downarrow$ results on real-world images.