Comparison of Different Deep Neural Network Models in the Cultural Heritage Domain

TL;DR

The paper tackles monument recognition in the cultural heritage domain by comparing CNN and Transformer architectures pre-trained on ImageNet and fine-tuned on a small Pisa dataset. It evaluates transferability and computational efficiency across VGG-11, ResNet-34, DenseNet-121, ViT-S, Swin-T, and PoolFormer-S24 using 20 randomized trials. Results show Swin-T achieves the best test accuracy, while DenseNet-121 provides similar validation performance with substantially lower computational cost, highlighting a trade-off between accuracy and efficiency. The work demonstrates the practicality of applying diverse DL architectures to a data-scarce cultural heritage task and points to Hebbian pre-training as a promising avenue for further improving data efficiency.

Abstract

The integration of computer vision and deep learning is an essential part of documenting and preserving cultural heritage, as well as improving visitor experiences. In recent years, two deep learning paradigms have been established in the field of computer vision: convolutional neural networks and transformer architectures. The present study aims to make a comparative analysis of some representatives of these two techniques of their ability to transfer knowledge from generic dataset, such as ImageNet, to cultural heritage specific tasks. The results of testing examples of the architectures VGG, ResNet, DenseNet, Visual Transformer, Swin Transformer, and PoolFormer, showed that DenseNet is the best in terms of efficiency-computability ratio.

Figures (3)

• Figure 1: Examples from the Pisa dataset.
• Figure 2: Accuracy measured on the testing and validation data for each fine-tuned network, shown in blue, compared to the k-nearest neighbour baseline classification, shown in orange, for all 20 trails.
• Figure 3: Accuracy measured on the testing and validation data for each fine-tuned network for all 20 trails.