JDEC: JPEG Decoding via Enhanced Continuous Cosine Coefficients

TL;DR

JDEC tackles JPEG artifact removal by decoding directly from compressed spectra using a local implicit neural representation with a continuous cosine spectrum estimator. It employs a SwinV2-based encoder with group spectra embedding and a continuous cosine formulation to infer dominant frequencies and amplitudes of the spectrum, enabling high-quality RGB reconstruction via an INR decoder. The single model handles multiple quality factors $q \in [10,100]$ without conventional JPEG decoding, achieving state-of-the-art results on LIVE-1, BSDS500, and ICB datasets, and introducing JDEC+ for extreme compression ($q=0$). This approach offers a practical, spectrum-driven JPEG decoder with favorable performance and flexible applicability to JPEG artifact removal.

Abstract

We propose a practical approach to JPEG image decoding, utilizing a local implicit neural representation with continuous cosine formulation. The JPEG algorithm significantly quantizes discrete cosine transform (DCT) spectra to achieve a high compression rate, inevitably resulting in quality degradation while encoding an image. We have designed a continuous cosine spectrum estimator to address the quality degradation issue that restores the distorted spectrum. By leveraging local DCT formulations, our network has the privilege to exploit dequantization and upsampling simultaneously. Our proposed model enables decoding compressed images directly across different quality factors using a single pre-trained model without relying on a conventional JPEG decoder. As a result, our proposed network achieves state-of-the-art performance in flexible color image JPEG artifact removal tasks. Our source code is available at https://github.com/WooKyoungHan/JDEC.

Figures (14)

• Figure 1: Overall concept of proposed JPEG decoding Instead of using a conventional JPEG decoder to refine the high-quality (HQ) image from the low-quality (LQ) image, our JDEC directly decodes the LQ spectrum by learning a continuous spectrum.
• Figure 2: Visual Demonstration at $q=100$ (PSNR (dB) $\uparrow$ / Bit-Error-Rate (BER) $\downarrow$) of decoding compressed image: JPEG (quality factor = 100), image enhancement approach fbcnn predicted from JPEG image ($q=100$), and JDEC (ours) predicted directly from a JPEG bit-stream. We highlight the occurrence of bit errors overlaid with green dots.
• Figure 3: Overall process of the JPEG encoder. Luminance and chroma components are separated from an RGB image. Both components are converted to DCT spectra and quantized with a pre-defined quantization matrix (Q-map). All losses occur in the orange area.
• Figure 4: Decoding a JPEG bitstream with the proposed JDEC. JDEC consists of an encoder ($E_\varphi$) with group spectra embedding ($g_\phi$), a decoder ($f_\theta$), and continuous cosine formulation ($T_\psi$). Inputs of JDEC are as follows: compressed spectra ($\mathbf{\tilde{X}_Y},{\mathbf{\tilde{X}_C}}$), quantization map $\mathbf{Q}$. Note that our JDEC does not take ${\mathbf{\tilde{I}}}$ as an input. JDEC formulates latent features into a trainable continuous cosine coefficient as a function of block grid $\delta$ and forward to INR ($f_\theta$). Therefore, each $B\times B$ block shares the estimated continuous cosine spectrum.
• Figure 5: Graphical summary of $f_\theta( T_\psi (\delta, \mathbf{z} ;\mathbf{Q}))$. Each $1\times1$-sized feature $\mathbf{z}$ maps into a $B\times B$ pixel area. $T_\psi$ embeds the local coordinates of $B\times B$ area and forwards to $f_\theta$.