Large-image Object Detection for Fine-grained Recognition of Punches Patterns in Medieval Panel Painting

TL;DR

This paper targets quantitative support for medieval art attribution by detecting and localizing punched patterns on large, high-resolution panel paintings. It advances a sliding-window object-detection pipeline based on YOLOv10 to identify punchmarks across 27 categories, using 8 ultra-high-resolution paintings and approximately 70,000 frames for training. A novel IoM-based non-maximal suppression merges predictions from overlapping windows, achieving high precision (up to about 0.94 on held-out data in the abstract) and robust F1 scores, thereby offering art historians a reproducible, scalable tool for attribution. The work also provides detailed dataset preparation, hyperparameter tuning, and open-source code to enable further development and broader application in art analysis.

Abstract

The attribution of the author of an art piece is typically a laborious manual process, usually relying on subjective evaluations of expert figures. However, there are some situations in which quantitative features of the artwork can support these evaluations. The extraction of these features can sometimes be automated, for instance, with the use of Machine Learning (ML) techniques. An example of these features is represented by repeated, mechanically impressed patterns, called punches, present chiefly in 13th and 14th-century panel paintings from Tuscany. Previous research in art history showcased a strong connection between the shapes of punches and specific artists or workshops, suggesting the possibility of using these quantitative cues to support the attribution. In the present work, we first collect a dataset of large-scale images of these panel paintings. Then, using YOLOv10, a recent and popular object detection model, we train a ML pipeline to perform object detection on the punches contained in the images. Due to the large size of the images, the detection procedure is split across multiple frames by adopting a sliding-window approach with overlaps, after which the predictions are combined for the whole image using a custom non-maximal suppression routine. Our results indicate how art historians working in the field can reliably use our method for the identification and extraction of punches.

Figures (7)

• Figure 1: Composition depicting the 8 artworks composing the dataset. The pictures represent the paintings at a variable scale.
• Figure 2: Samples from the punches in our dataset, one per category.
• Figure 3: Crops of the high-resolution images of paintings showcasing some of the punchmarks after the labelling procedure.
• Figure 4: Barcharts depicting the per-category distribution of the punchmarks in our dataset. (a) The distribution of the original dataset before preprocessing. (b) The distribution after the preprocessing and before rebalancing. (c) The distribution after rebalancing.
• Figure 5: Illustration of the procedure we operated for splitting the dataset into training and validation splits. We divide the image into square grids of equal size (at least 2160 px per side, depending on the full-resolution image size). The frames are separated by gutters (depicted in yellow) in order to avoid a single frame to leak onto two different data splits. The cells coloured in blue are assigned to the training set, while those depicted in green are allocated to the validation set. The red squares represent a possible configuration of frames obtained by the random sampling procedure.