Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Novel Artistic Scene-Centric Datasets for Effective Transfer Learning in Fragrant Spaces

Shumei Liu, Haiting Huang, Mathias Zinnen, Andreas Maier, Vincent Christlein

TL;DR

The study tackles the lack of annotated fragrant-space data in historical artworks by proposing a weakly labeled transfer-learning approach that leverages Places365 pretraining. It builds the ArtPlaces dataset from Rijksmuseum (RASD), Wikidata (WASD), and Fragrant-Places, with 3804 weakly labeled training images and 975 manually labeled test images, plus a Fragrant-Spaces test of 228 images. Models pre-trained on Places365 are fine-tuned on ArtPlaces-train and evaluated on Fragrant-Spaces and ArtPlaces-test across four architectures, with DenseNet161 frequently delivering the best results (e.g., top-1 ≈24.12% on Fragrant-Spaces and ≈29.73% on ArtPlaces-test). The results demonstrate that weak-label transfer learning substantially boosts performance in both fragrant-space recognition and broader artistic scene classification, while highlighting limitations due to dataset size and label quality, and pointing toward scaling up data sources for further improvements.

Abstract

Olfaction, often overlooked in cultural heritage studies, holds profound significance in shaping human experiences and identities. Examining historical depictions of olfactory scenes can offer valuable insights into the role of smells in history. We show that a transfer-learning approach using weakly labeled training data can remarkably improve the classification of fragrant spaces and, more generally, artistic scene depictions. We fine-tune Places365-pre-trained models by querying two cultural heritage data sources and using the search terms as supervision signal. The models are evaluated on two manually corrected test splits. This work lays a foundation for further exploration of fragrant spaces recognition and artistic scene classification. All images and labels are released as the ArtPlaces dataset at https://zenodo.org/doi/10.5281/zenodo.11584328.

Novel Artistic Scene-Centric Datasets for Effective Transfer Learning in Fragrant Spaces

TL;DR

The study tackles the lack of annotated fragrant-space data in historical artworks by proposing a weakly labeled transfer-learning approach that leverages Places365 pretraining. It builds the ArtPlaces dataset from Rijksmuseum (RASD), Wikidata (WASD), and Fragrant-Places, with 3804 weakly labeled training images and 975 manually labeled test images, plus a Fragrant-Spaces test of 228 images. Models pre-trained on Places365 are fine-tuned on ArtPlaces-train and evaluated on Fragrant-Spaces and ArtPlaces-test across four architectures, with DenseNet161 frequently delivering the best results (e.g., top-1 ≈24.12% on Fragrant-Spaces and ≈29.73% on ArtPlaces-test). The results demonstrate that weak-label transfer learning substantially boosts performance in both fragrant-space recognition and broader artistic scene classification, while highlighting limitations due to dataset size and label quality, and pointing toward scaling up data sources for further improvements.

Abstract

Olfaction, often overlooked in cultural heritage studies, holds profound significance in shaping human experiences and identities. Examining historical depictions of olfactory scenes can offer valuable insights into the role of smells in history. We show that a transfer-learning approach using weakly labeled training data can remarkably improve the classification of fragrant spaces and, more generally, artistic scene depictions. We fine-tune Places365-pre-trained models by querying two cultural heritage data sources and using the search terms as supervision signal. The models are evaluated on two manually corrected test splits. This work lays a foundation for further exploration of fragrant spaces recognition and artistic scene classification. All images and labels are released as the ArtPlaces dataset at https://zenodo.org/doi/10.5281/zenodo.11584328.
Paper Structure (21 sections, 5 figures, 8 tables)

This paper contains 21 sections, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. The ArtPlaces Dataset
  3. Pre-Training Data
  4. Data Collection
  5. Artistic Scene-Centric Datasets.
  6. RASD.
  7. WASD.
  8. Fragrant-Places.
  9. Final Data Split.
  10. Source Data Analysis
  11. Label Quality & Correction
  12. Quality Evaluation.
  13. Error Analysis.
  14. Data Rectification.
  15. Methodology
  16. ...and 6 more sections

Figures (5)

  • Figure 1: Image samples of the artistic scene-centric datasets.
  • Figure 2: Image samples of the Fragrant-Spaces dataset.
  • Figure 3: Illustration of the dataset creation process. ArtPlaces-raw is derived from the weakly labeled RASD and WASD subsets. From ArtPlaces-raw, ArtPlaces-train is split, and in conjunction with Fragrant-Spaces ArtPlaces-test.
  • Figure 4: Dominant categories in different datasets, contributing separately 90% of each data volume.
  • Figure 5: Comparison of the number of images per category between datasets RASD, WASD, and Fragrant-Spaces. In each polar coordinate, the radial axis measures the number of images per category, and the angular axis represents the sorted dominant categories.