Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prominence-Aware Artifact Detection and Dataset for Image Super-Resolution

Ivan Molodetskikh, Kirill Malyshev, Mark Mirgaleev, Nikita Zagainov, Evgeney Bogatyrev, Dmitriy Vatolin

TL;DR

The paper tackles perceptual artifacts in SR by introducing prominence-aware artifact detection and a dataset with crowdsourced prominence annotations for 1302 SR artifact examples from 11 methods. It proposes a lightweight MLP-based prominence regressor that combines three features to produce artifact prominence heatmaps, and demonstrates the feasibility of using real-time SR as pseudo-ground-truth for full-reference metrics. The authors show that prominence-aware evaluation yields more robust detection of disturbing artifacts and enable effective fine-tuning of SR models to reduce artifacts, with practical release of code and datasets. By highlighting artifact prominence, the work provides a path toward perceptually aligned SR evaluation and mitigation in real-world settings where ground-truth HR images are unavailable.

Abstract

Generative image super-resolution (SR) is rapidly advancing in visual quality and detail restoration. As the capacity of SR models expands, however, so does their tendency to produce artifacts: incorrect, visually disturbing details that reduce perceived quality. Crucially, their perceptual impact varies: some artifacts are barely noticeable while others strongly degrade the image. We argue that artifacts should be characterized by their prominence to human observers rather than treated as uniform binary defects. Motivated by this, we present a novel dataset of 1302 artifact examples from 11 contemporary image-SR methods, where each artifact is paired with a crowdsourced prominence score. Building on this dataset, we train a lightweight regressor that produces spatial prominence heatmaps and outperforms existing methods at detecting prominent artifacts. We release the dataset and code to facilitate prominence-aware evaluation and mitigation of SR artifacts.

Prominence-Aware Artifact Detection and Dataset for Image Super-Resolution

TL;DR

The paper tackles perceptual artifacts in SR by introducing prominence-aware artifact detection and a dataset with crowdsourced prominence annotations for 1302 SR artifact examples from 11 methods. It proposes a lightweight MLP-based prominence regressor that combines three features to produce artifact prominence heatmaps, and demonstrates the feasibility of using real-time SR as pseudo-ground-truth for full-reference metrics. The authors show that prominence-aware evaluation yields more robust detection of disturbing artifacts and enable effective fine-tuning of SR models to reduce artifacts, with practical release of code and datasets. By highlighting artifact prominence, the work provides a path toward perceptually aligned SR evaluation and mitigation in real-world settings where ground-truth HR images are unavailable.

Abstract

Generative image super-resolution (SR) is rapidly advancing in visual quality and detail restoration. As the capacity of SR models expands, however, so does their tendency to produce artifacts: incorrect, visually disturbing details that reduce perceived quality. Crucially, their perceptual impact varies: some artifacts are barely noticeable while others strongly degrade the image. We argue that artifacts should be characterized by their prominence to human observers rather than treated as uniform binary defects. Motivated by this, we present a novel dataset of 1302 artifact examples from 11 contemporary image-SR methods, where each artifact is paired with a crowdsourced prominence score. Building on this dataset, we train a lightweight regressor that produces spatial prominence heatmaps and outperforms existing methods at detecting prominent artifacts. We release the dataset and code to facilitate prominence-aware evaluation and mitigation of SR artifacts.
Paper Structure (33 sections, 2 equations, 14 figures, 23 tables)

This paper contains 33 sections, 2 equations, 14 figures, 23 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Artifact dataset
  4. Crowdsourced annotation setup
  5. Mask postprocessing
  6. Dataset analysis
  7. Artifact prominence metric
  8. Input features
  9. Prominence regressor
  10. Adapting full-resolution metrics with real-time-SR pseudo-GT
  11. Training
  12. Experiments
  13. Objective evaluation methodology
  14. Objective prominence-metric evaluation
  15. Evaluating robustness to SR artifacts
  16. ...and 18 more sections

Figures (14)

  • Figure 1: Examples of prominent artifacts detected by the proposed method. Top: example from our proposed dataset. GFPGAN failed to restore holes on the radio panel. Bottom: example from the DeSRA dataset. LDL produced an irregular line pattern on the carpet. Prominence denotes the percentage of annotators confirming the artifact in the highlighted area (\ref{['sec:dataset']}).
  • Figure 2: Non-prominent artifacts detected by existing methods. Top: example from our proposed dataset. The water surface was incorrectly restored by GFPGAN producing an artifact, but it's a natural surface, so this artifact is not prominent to humans. Bottom: example from the DeSRA dataset. SwinIR produced a texture artifact (vertical lines) on the floor, but it's in a non-salient region, so it's not prominent to humans. Our method correctly marks both examples with low prominence.
  • Figure 3: Example of our postprocessing technique increasing artifact visibility. On the left is the original mask from the DeSRA dataset; on the right is the mask after postprocessing.
  • Figure 4: Prominence distribution for our dataset (based on Open Images) and for the DeSRA dataset (for which we collected prominence annotations).
  • Figure 5: Interface presented to the viewers during subjective data collection.
  • ...and 9 more figures