Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BiblioPage: A Dataset of Scanned Title Pages for Bibliographic Metadata Extraction

Jan Kohút, Martin Dočekal, Michal Hradiš, Marek Vaško

TL;DR

BiblioPage addresses the bottleneck of manually digitizing bibliographic metadata by introducing a dataset of ~2,118 scanned title pages from 14 Czech libraries, annotated with 16 bibliographic attributes and precise bounding boxes. The work establishes two robust evaluation pipelines—object-detection with OCR and Vision-Language Large Models (VLLMs)—and benchmarked them on this real-world corpus, revealing complementary strengths: detectors perform well on visually distinctive fields while VLLMs leverage textual semantics, though titles/subtitles remain particularly challenging. Key findings include a maximum mAP of 52 and an F1 score of 59 for detection-based methods, and an F1 score of 67 (rising to 70 with OCR) for GPT-4o in VLLM-based runs. The dataset and accompanying evaluation scripts provide a practical benchmark for document understanding, document question answering, and information extraction in archives, enabling development of automated bibliographic metadata extraction tools tailored to historical and heterogeneous layouts.

Abstract

Manual digitization of bibliographic metadata is time consuming and labor intensive, especially for historical and real-world archives with highly variable formatting across documents. Despite advances in machine learning, the absence of dedicated datasets for metadata extraction hinders automation. To address this gap, we introduce BiblioPage, a dataset of scanned title pages annotated with structured bibliographic metadata. The dataset consists of approximately 2,000 monograph title pages collected from 14 Czech libraries, spanning a wide range of publication periods, typographic styles, and layout structures. Each title page is annotated with 16 bibliographic attributes, including title, contributors, and publication metadata, along with precise positional information in the form of bounding boxes. To extract structured information from this dataset, we valuated object detection models such as YOLO and DETR combined with transformer-based OCR, achieving a maximum mAP of 52 and an F1 score of 59. Additionally, we assess the performance of various visual large language models, including LlamA 3.2-Vision and GPT-4o, with the best model reaching an F1 score of 67. BiblioPage serves as a real-world benchmark for bibliographic metadata extraction, contributing to document understanding, document question answering, and document information extraction. Dataset and evaluation scripts are availible at: https://github.com/DCGM/biblio-dataset

BiblioPage: A Dataset of Scanned Title Pages for Bibliographic Metadata Extraction

TL;DR

BiblioPage addresses the bottleneck of manually digitizing bibliographic metadata by introducing a dataset of ~2,118 scanned title pages from 14 Czech libraries, annotated with 16 bibliographic attributes and precise bounding boxes. The work establishes two robust evaluation pipelines—object-detection with OCR and Vision-Language Large Models (VLLMs)—and benchmarked them on this real-world corpus, revealing complementary strengths: detectors perform well on visually distinctive fields while VLLMs leverage textual semantics, though titles/subtitles remain particularly challenging. Key findings include a maximum mAP of 52 and an F1 score of 59 for detection-based methods, and an F1 score of 67 (rising to 70 with OCR) for GPT-4o in VLLM-based runs. The dataset and accompanying evaluation scripts provide a practical benchmark for document understanding, document question answering, and information extraction in archives, enabling development of automated bibliographic metadata extraction tools tailored to historical and heterogeneous layouts.

Abstract

Manual digitization of bibliographic metadata is time consuming and labor intensive, especially for historical and real-world archives with highly variable formatting across documents. Despite advances in machine learning, the absence of dedicated datasets for metadata extraction hinders automation. To address this gap, we introduce BiblioPage, a dataset of scanned title pages annotated with structured bibliographic metadata. The dataset consists of approximately 2,000 monograph title pages collected from 14 Czech libraries, spanning a wide range of publication periods, typographic styles, and layout structures. Each title page is annotated with 16 bibliographic attributes, including title, contributors, and publication metadata, along with precise positional information in the form of bounding boxes. To extract structured information from this dataset, we valuated object detection models such as YOLO and DETR combined with transformer-based OCR, achieving a maximum mAP of 52 and an F1 score of 59. Additionally, we assess the performance of various visual large language models, including LlamA 3.2-Vision and GPT-4o, with the best model reaching an F1 score of 67. BiblioPage serves as a real-world benchmark for bibliographic metadata extraction, contributing to document understanding, document question answering, and document information extraction. Dataset and evaluation scripts are availible at: https://github.com/DCGM/biblio-dataset

Paper Structure

This paper contains 16 sections, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset
  4. Data Sourcing and Distribution
  5. Annotation Process
  6. Attribute annotation.
  7. Text extraction.
  8. Aligning attributes to the extracted text.
  9. Annotation rules for attributes.
  10. Evaluation protocol
  11. Baselines
  12. Object detection
  13. DETR.
  14. VLLM
  15. Conslusion
  16. ...and 1 more sections

Figures (6)

  • Figure 1: Representative title pages from BiblioPage dataset, selected to illustrate the various types found within it.
  • Figure 2: The distribution of BiblioPage dataset documents' over libraries.
  • Figure 3: The distribution of BibliPage dataset documents' issue dates over time.
  • Figure 4: Attribute distribution in the BiblioPage dataset for the development and test sets. A logarithmic scale is applied to the x-axis for visualization purposes.
  • Figure 5: Automatic alignment of bibliographic information and its correction (left), automatic layout detection and its correction (right).
  • ...and 1 more figures