Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EUFCC-340K: A Faceted Hierarchical Dataset for Metadata Annotation in GLAM Collections

Francesc Net, Marc Folia, Pep Casals, Andrew D. Bagdanov, Lluis Gomez

TL;DR

GLAMs face scalability bottlenecks in metadata annotation; EUFCC-340K provides a large, AAT-aligned, four-facet hierarchical dataset derived from Europeana to support automatic image tagging. The authors propose CNN-based multi-head and CLIP-based baselines that operate across Materials, Object Types, Disciplines, and Subjects, and evaluate them under two test splits to simulate intra-domain and cross-domain generalization. They show that ensemble approaches improve annotation quality and introduce an annotation assistant tool to streamline cataloging, though outer-domain generalization remains challenging. The dataset and tools are publicly released to propel research in automated GLAM metadata annotation.

Abstract

In this paper, we address the challenges of automatic metadata annotation in the domain of Galleries, Libraries, Archives, and Museums (GLAMs) by introducing a novel dataset, EUFCC340K, collected from the Europeana portal. Comprising over 340,000 images, the EUFCC340K dataset is organized across multiple facets: Materials, Object Types, Disciplines, and Subjects, following a hierarchical structure based on the Art & Architecture Thesaurus (AAT). We developed several baseline models, incorporating multiple heads on a ConvNeXT backbone for multi-label image tagging on these facets, and fine-tuning a CLIP model with our image text pairs. Our experiments to evaluate model robustness and generalization capabilities in two different test scenarios demonstrate the utility of the dataset in improving multi-label classification tools that have the potential to alleviate cataloging tasks in the cultural heritage sector.

EUFCC-340K: A Faceted Hierarchical Dataset for Metadata Annotation in GLAM Collections

TL;DR

GLAMs face scalability bottlenecks in metadata annotation; EUFCC-340K provides a large, AAT-aligned, four-facet hierarchical dataset derived from Europeana to support automatic image tagging. The authors propose CNN-based multi-head and CLIP-based baselines that operate across Materials, Object Types, Disciplines, and Subjects, and evaluate them under two test splits to simulate intra-domain and cross-domain generalization. They show that ensemble approaches improve annotation quality and introduce an annotation assistant tool to streamline cataloging, though outer-domain generalization remains challenging. The dataset and tools are publicly released to propel research in automated GLAM metadata annotation.

Abstract

In this paper, we address the challenges of automatic metadata annotation in the domain of Galleries, Libraries, Archives, and Museums (GLAMs) by introducing a novel dataset, EUFCC340K, collected from the Europeana portal. Comprising over 340,000 images, the EUFCC340K dataset is organized across multiple facets: Materials, Object Types, Disciplines, and Subjects, following a hierarchical structure based on the Art & Architecture Thesaurus (AAT). We developed several baseline models, incorporating multiple heads on a ConvNeXT backbone for multi-label image tagging on these facets, and fine-tuning a CLIP model with our image text pairs. Our experiments to evaluate model robustness and generalization capabilities in two different test scenarios demonstrate the utility of the dataset in improving multi-label classification tools that have the potential to alleviate cataloging tasks in the cultural heritage sector.
Paper Structure (16 sections, 3 equations, 13 figures, 3 tables)

This paper contains 16 sections, 3 equations, 13 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The EUFCC-340K Dataset
  4. Baseline Methods
  5. CNN-based vision models
  6. Multi-label classification
  7. Softmax classification
  8. Hierarchical Softmax
  9. Multi-modal baselines
  10. Experiments
  11. Evaluation Metrics
  12. Implementation details
  13. Automatic metadata annotation results
  14. Qualitative results
  15. An annotation assistant tool for GLAM cataloguers
  16. ...and 1 more sections

Figures (13)

  • Figure 1: Sample records from the EUFCC-340K dataset. Each image in the dataset is annotated across different facets of the Getty "Art & Architecture Thesaurus" (Some nodes are omitted for visualization purposes in this figure).
  • Figure 2: Description and hierarchical position of the AAT term "Numismatics" according to the AAT Online search tool.
  • Figure 3: Items' distribution for the $30$ most frequent data providers within the EUFCC-340K dataset.
  • Figure 4: EUFCC-340K statistics: tag count and tag length variation per image.
  • Figure 5: Sample records from the EUFCC-340K dataset. Each image in the dataset is annotated across different facets' hierarchies of the Getty "Art & Architecture Thesaurus". Some nodes were omitted for visualization purposes.
  • ...and 8 more figures