Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

State-of-the-Art Fails in the Art of Damage Detection

Daniela Ivanova, Marco Aversa, Paul Henderson, John Williamson

TL;DR

Analogue-media damage detection is crucial for preservation but challenging to generalize across media types. The authors introduce DamBench, a large, richly annotated dataset with 418 high-resolution images, 11,000 pixel-level damage masks, 15 damage types, and material/content labels. They evaluate supervised semantic segmentation and text-guided diffusion-based segmentation, revealing substantial generalization gaps across media and damage types. The results underscore the need for multimodal, cross-domain approaches to reliably detect damage in diverse analogue artifacts.

Abstract

Accurately detecting and classifying damage in analogue media such as paintings, photographs, textiles, mosaics, and frescoes is essential for cultural heritage preservation. While machine learning models excel in correcting global degradation if the damage operator is known a priori, we show that they fail to predict where the damage is even after supervised training; thus, reliable damage detection remains a challenge. We introduce DamBench, a dataset for damage detection in diverse analogue media, with over 11,000 annotations covering 15 damage types across various subjects and media. We evaluate CNN, Transformer, and text-guided diffusion segmentation models, revealing their limitations in generalising across media types.

State-of-the-Art Fails in the Art of Damage Detection

TL;DR

Analogue-media damage detection is crucial for preservation but challenging to generalize across media types. The authors introduce DamBench, a large, richly annotated dataset with 418 high-resolution images, 11,000 pixel-level damage masks, 15 damage types, and material/content labels. They evaluate supervised semantic segmentation and text-guided diffusion-based segmentation, revealing substantial generalization gaps across media and damage types. The results underscore the need for multimodal, cross-domain approaches to reliably detect damage in diverse analogue artifacts.

Abstract

Accurately detecting and classifying damage in analogue media such as paintings, photographs, textiles, mosaics, and frescoes is essential for cultural heritage preservation. While machine learning models excel in correcting global degradation if the damage operator is known a priori, we show that they fail to predict where the damage is even after supervised training; thus, reliable damage detection remains a challenge. We introduce DamBench, a dataset for damage detection in diverse analogue media, with over 11,000 annotations covering 15 damage types across various subjects and media. We evaluate CNN, Transformer, and text-guided diffusion segmentation models, revealing their limitations in generalising across media types.
Paper Structure (11 sections, 2 figures, 1 table)

This paper contains 11 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Existing work on analogue media damage.
  3. Damaged Analogue Media Dataset
  4. Damage types.
  5. Material categorisation.
  6. Content categorisation.
  7. Textual descriptions.
  8. Benchmark
  9. Supervised semantic segmentation.
  10. Text-guided segmentation.
  11. Results.

Figures (2)

  • Figure 1: Examples from our dataset of damaged artwork, categorised by Material (rows 1 and 2) and Content (row 3). Annotation colours correspond to different types of damage. Note the diversity of media and content, and pixel-accurate damage masks.
  • Figure 2: Qualitative comparison for binary damage segmentation.