Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Comparison of Feature Learning Methods for Metadata Extraction from PDF Scholarly Documents

Zeyd Boukhers, Cong Yang

TL;DR

This work tackles metadata extraction from PDF scholarly documents under wide template variability, a key facet of FAIR principles. It systematically compares NLP, CV, and multimodal methods, including CRF, BiLSTM-based models, GROBID, Fast-RCNN, and the proposed TextMap framework, across two challenging datasets. The results show that multimodal TextMap variants, especially TextMap with Word2Vec and with BERT embeddings, achieve top performance (e.g., F1 up to 0.913), while traditional methods remain competitive for structured fields; the study also highlights trade-offs between accuracy and computational cost. The paper provides practical guidance for selecting approaches based on document diversity and resource constraints and shares implementations to support reproducibility and further research.

Abstract

The availability of metadata for scientific documents is pivotal in propelling scientific knowledge forward and for adhering to the FAIR principles (i.e. Findability, Accessibility, Interoperability, and Reusability) of research findings. However, the lack of sufficient metadata in published documents, particularly those from smaller and mid-sized publishers, hinders their accessibility. This issue is widespread in some disciplines, such as the German Social Sciences, where publications often employ diverse templates. To address this challenge, our study evaluates various feature learning and prediction methods, including natural language processing (NLP), computer vision (CV), and multimodal approaches, for extracting metadata from documents with high template variance. We aim to improve the accessibility of scientific documents and facilitate their wider use. To support our comparison of these methods, we provide comprehensive experimental results, analyzing their accuracy and efficiency in extracting metadata. Additionally, we provide valuable insights into the strengths and weaknesses of various feature learning and prediction methods, which can guide future research in this field.

Comparison of Feature Learning Methods for Metadata Extraction from PDF Scholarly Documents

TL;DR

This work tackles metadata extraction from PDF scholarly documents under wide template variability, a key facet of FAIR principles. It systematically compares NLP, CV, and multimodal methods, including CRF, BiLSTM-based models, GROBID, Fast-RCNN, and the proposed TextMap framework, across two challenging datasets. The results show that multimodal TextMap variants, especially TextMap with Word2Vec and with BERT embeddings, achieve top performance (e.g., F1 up to 0.913), while traditional methods remain competitive for structured fields; the study also highlights trade-offs between accuracy and computational cost. The paper provides practical guidance for selecting approaches based on document diversity and resource constraints and shares implementations to support reproducibility and further research.

Abstract

The availability of metadata for scientific documents is pivotal in propelling scientific knowledge forward and for adhering to the FAIR principles (i.e. Findability, Accessibility, Interoperability, and Reusability) of research findings. However, the lack of sufficient metadata in published documents, particularly those from smaller and mid-sized publishers, hinders their accessibility. This issue is widespread in some disciplines, such as the German Social Sciences, where publications often employ diverse templates. To address this challenge, our study evaluates various feature learning and prediction methods, including natural language processing (NLP), computer vision (CV), and multimodal approaches, for extracting metadata from documents with high template variance. We aim to improve the accessibility of scientific documents and facilitate their wider use. To support our comparison of these methods, we provide comprehensive experimental results, analyzing their accuracy and efficiency in extracting metadata. Additionally, we provide valuable insights into the strengths and weaknesses of various feature learning and prediction methods, which can guide future research in this field.
Paper Structure (32 sections, 24 equations, 7 figures, 13 tables)

This paper contains 32 sections, 24 equations, 7 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Natural Language Processing
  4. Computer Vision
  5. Multimodality
  6. Approach
  7. Conditional Random Fields (CRF)souza2014arctic
  8. Bi-Directional LSTM
  9. BiLSTM-CRF
  10. GrobidGROBID
  11. Fast-RCNN
  12. Vision and Natural Language
  13. Natural Language-based Model
  14. Computer Vision-based Model
  15. Classifier
  16. ...and 17 more sections

Figures (7)

  • Figure 1: Schematic representation of the two-layer Conditional Random Field (CRF) model for metadata extraction souza2014arctic.
  • Figure 2: Diagram of the Bi-Directional LSTM network architecture for metadata extraction
  • Figure 3: Diagram of the Bi-Directional LSTM network architecture with CRF classifier for metadata extraction
  • Figure 4: Overview of the GROBID Framework for structured metadata extraction GROBID
  • Figure 5: Mask R-CNN architecture employed for metadata extraction from PDF pages.
  • ...and 2 more figures