Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifacts

TL;DR

This work tackles artifact-bearing HF hallucinations in single-image SR by steering optimization in the wavelet domain. It introduces WGSR, a GAN-SR framework that trains with SWT-domain fidelity losses and uses an HF-subband discriminator to better separate genuine details from artifacts, achieving improved perception-distortion balance. Empirical results across standard benchmarks show enhanced NRQM and competitive PSNR, with qualitative gains in preserving fine structures and reducing HF artifacts. The approach is presented as a general, plug-in strategy for artifact control in SR that aligns optimization with multi-scale, directional image structures.

Abstract

Super-resolution (SR) is an ill-posed inverse problem, where the size of the set of feasible solutions that are consistent with a given low-resolution image is very large. Many algorithms have been proposed to find a "good" solution among the feasible solutions that strike a balance between fidelity and perceptual quality. Unfortunately, all known methods generate artifacts and hallucinations while trying to reconstruct high-frequency (HF) image details. A fundamental question is: Can a model learn to distinguish genuine image details from artifacts? Although some recent works focused on the differentiation of details and artifacts, this is a very challenging problem and a satisfactory solution is yet to be found. This paper shows that the characterization of genuine HF details versus artifacts can be better learned by training GAN-based SR models using wavelet-domain loss functions compared to RGB-domain or Fourier-space losses. Although wavelet-domain losses have been used in the literature before, they have not been used in the context of the SR task. More specifically, we train the discriminator only on the HF wavelet sub-bands instead of on RGB images and the generator is trained by a fidelity loss over wavelet subbands to make it sensitive to the scale and orientation of structures. Extensive experimental results demonstrate that our model achieves better perception-distortion trade-off according to multiple objective measures and visual evaluations.

Figures (7)

• Figure 1: Perception-distortion trade-off performance of our model WGSR vs. state-of-the-art methods on the PSNR-NRQM plane. Dashed curve shows the theoretical limit explained in Blau_2018.
• Figure 2: Visual performance of recent $\times$4 SR methods on a crop from Urban100 dataset (img-6) urban100_cite. SOTA methods reconstruct "5" as "6", whereas the opening in the lower part of "5" is visible in our result confirming that our model strikes a better balance between fidelity and visual quality. Note PSNR, DISTS and other quantitative scores are not good indicators of such artifacts.
• Figure 3: Overview of the proposed GAN-SR framework guided by wavelet-domain losses, where the strength of the adversarial loss is tuned for each subband to control artifacts and the discriminator learns to decide whether the generated detail subbands are real or fake.
• Figure 4: Illustration of our main premise that imposing different losses to different SWT subbands results in remarkable quantitative and qualitative performance improvements in GAN-based SR models. Specifically, enforcing fidelity loss on wavelet sub-bands instead of on RGB channels and running the discriminator only on detail (LH, HL, and HH) subbands helps eliminate visible artifacts caused by ESRGAN+ esrganplus and leads to better preservation of details. Overall scores of our method WGSR (PSNR: 26.33/DISTS: 0.115) outperform ESRGAN+ esrganplus (PSNR: 22.78/DISTS: 0.225).
• Figure 5: Visual comparison of the proposed wavelet-guided perceptual optimization method with the state-of-the-art methods for $\times$4 SR on natural images from BSD100 validation set. Our method WGSR with 2-level SWT provides the best balance between perception-distortion trade-off for natural images and it has clear advantages in reconstructing realistic HF details while inhibiting artifacts. Additional visual comparisons can be found in the supplementary materials.