Drawing the Line: Deep Segmentation for Extracting Art from Ancient Etruscan Mirrors

TL;DR

Automatic segmentation of engravings on the backs of Etruscan hand mirrors is addressed using photometric stereo data and patch-based deep segmentation to mitigate damage-induced subjectivity. The approach combines an EfficientNet-B6 encoder with a UNet decoder, patch-wise training, and augmentations, including self-supervised Cross-Pseudo Supervision (CPS) experiments. It achieves a ~16% improvement in pseudo-F-Measure over a baseline and attains human-level performance on complete mirrors, outperforming binarization baselines. The work advances cultural heritage analysis by increasing annotation objectivity and efficiency, and it releases code and data for reproducibility.

Abstract

Etruscan mirrors constitute a significant category within Etruscan art and, therefore, undergo systematic examinations to obtain insights into ancient times. A crucial aspect of their analysis involves the labor-intensive task of manually tracing engravings from the backside. Additionally, this task is inherently challenging due to the damage these mirrors have sustained, introducing subjectivity into the process. We address these challenges by automating the process through photometric-stereo scanning in conjunction with deep segmentation networks which, however, requires effective usage of the limited data at hand. We accomplish this by incorporating predictions on a per-patch level, and various data augmentations, as well as exploring self-supervised learning. Compared to our baseline, we improve predictive performance w.r.t. the pseudo-F-Measure by around 16%. When assessing performance on complete mirrors against a human baseline, our approach yields quantitative similar performance to a human annotator and significantly outperforms existing binarization methods. With our proposed methodology, we streamline the annotation process, enhance its objectivity, and reduce overall workload, offering a valuable contribution to the examination of these historical artifacts and other non-traditional documents.

Figures (12)

• Figure 1: A typical Etruscan mirror: fine drawings adorn its backside.
• Figure 2: Exemplary albedo, depth, normal map, and corresponding ground truth.
• Figure 3: Comparing the effect of preprocessing on the depth modality, i.e., employing a high-pass filter to remove low frequencies and to make lines more visible.
• Figure 4: In order to extract artistic lines from damaged Etruscan hand mirrors, we employ a deep segmentation network on a per-patch level: green denotes patches used during training, red, patches not used during training and blue, ones that are included within the current mini-batch.
• Figure 5: Showcasing our ablation study (\ref{['tab:ablation']}) using the exemplary mirror "ANSA-VI-1700", resulting in an IoU of 38.3 and a pFM of 57.6; the union of green and black denotes the prediction, red and black annotation A (\ref{['fig:A']}).