Do Transformers Understand Ancient Roman Coin Motifs Better than CNNs?

TL;DR

The paper tackles the problem of automatically identifying semantic motifs on ancient coins, shifting from image matching to semantic element recognition. It compares Vision Transformer (ViT) architectures with CNN baselines on a large, multi-modal dataset derived from coin images and descriptions, focusing on reverse-side motifs and using text-mined labels. ViT generally achieves higher accuracy across most concepts, though results are affected by noisy labels and variable saliency interpretations; the study also demonstrates the value and challenges of visual explanations with HiPe. The work demonstrates ViT’s promise for nuanced automated coin analysis and outlines concrete paths for improving labeling, data quality, and potential multi-modal modeling in numismatics.

Abstract

Automated analysis of ancient coins has the potential to help researchers extract more historical insights from large collections of coins and to help collectors understand what they are buying or selling. Recent research in this area has shown promise in focusing on identification of semantic elements as they are commonly depicted on ancient coins, by using convolutional neural networks (CNNs). This paper is the first to apply the recently proposed Vision Transformer (ViT) deep learning architecture to the task of identification of semantic elements on coins, using fully automatic learning from multi-modal data (images and unstructured text). This article summarises previous research in the area, discusses the training and implementation of ViT and CNN models for ancient coins analysis and provides an evaluation of their performance. The ViT models were found to outperform the newly trained CNN models in accuracy.

Figures (24)

• Figure 1: A high-level view of the ViT architecture, showing how it is built on top of a standard Transformer Encoder. Image source: Davide Coccomini. Licensed to share under CC-BY-SA-4.0.
• Figure 2: An example of a typical sample from the data set, consisting of an image that shows a coin's obverse on the left and its reverse on the right, and a description of the coin.
• Figure 3: Examples of some of the images that were automatically rejected during preprocessing.
• Figure 4: The algorithm successfully preprocessed the image on the left to the generate the one on the right.
• Figure 5: An example of an image that was not rejected during preprocessing but ideally would have been, as it includes multiple coins.