Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient OCR for Building a Diverse Digital History

Jacob Carlson, Tom Bryan, Melissa Dell

TL;DR

This work introduces EffOCR, a retrieval-based OCR framework that treats character and word recognition as image-to-embedding retrieval rather than sequence-to-sequence decoding with a language model. By localizing text regions and embedding crops in a contrastively learned metric space, decoding is achieved via nearest-neighbor search against an offline exemplar index, enabling strong performance with limited labeled data across diverse historical scripts. The authors demonstrate competitive or superior character error rates on Japanese, English, and Greek datasets, with substantial sample efficiency and dramatic CPU-inference speed advantages, making large-scale digitization of archives feasible and inviting community-driven extensions. The approach reduces reliance on language models, supports adding new characters at inference time, and aligns with goals of making digital history more representative of documentary history while maintaining open-source accessibility and low environmental impact.

Abstract

Thousands of users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) - which jointly learns a vision and language model - is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters' visual features, it is more sample efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, the model opens new avenues for community engagement in making digital history more representative of documentary history.

Efficient OCR for Building a Diverse Digital History

TL;DR

This work introduces EffOCR, a retrieval-based OCR framework that treats character and word recognition as image-to-embedding retrieval rather than sequence-to-sequence decoding with a language model. By localizing text regions and embedding crops in a contrastively learned metric space, decoding is achieved via nearest-neighbor search against an offline exemplar index, enabling strong performance with limited labeled data across diverse historical scripts. The authors demonstrate competitive or superior character error rates on Japanese, English, and Greek datasets, with substantial sample efficiency and dramatic CPU-inference speed advantages, making large-scale digitization of archives feasible and inviting community-driven extensions. The approach reduces reliance on language models, supports adding new characters at inference time, and aligns with goals of making digital history more representative of documentary history while maintaining open-source accessibility and low environmental impact.

Abstract

Thousands of users consult digital archives daily, but the information they can access is unrepresentative of the diversity of documentary history. The sequence-to-sequence architecture typically used for optical character recognition (OCR) - which jointly learns a vision and language model - is poorly extensible to low-resource document collections, as learning a language-vision model requires extensive labeled sequences and compute. This study models OCR as a character level image retrieval problem, using a contrastively trained vision encoder. Because the model only learns characters' visual features, it is more sample efficient and extensible than existing architectures, enabling accurate OCR in settings where existing solutions fail. Crucially, the model opens new avenues for community engagement in making digital history more representative of documentary history.
Paper Structure (10 sections, 1 equation, 5 figures, 3 tables)

This paper contains 10 sections, 1 equation, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Related Literature
  4. Training and Evaluation datasets
  5. Measurement and comparisons
  6. Results
  7. Discussion
  8. Reproducibility
  9. Limitations
  10. Ethical Considerations

Figures (5)

  • Figure 1: EffOCR and Seq2Seq Model Architectures. This figure represents the EffOCR architecture, as compared to a typical sequence-to-sequence OCR architecture.
  • Figure 2: Diversity in the Chronicling America Dataset. This figure shows examples sampled from the Chronicling America (LoCCA) dataset, along with EffOCR predicted transcriptions.
  • Figure 3: Error Analysis. Representative examples of EffOCR errors, showing the target crop, the EffOCR localized crop, and the five nearest characters in the embedding index, with the correct character highlighted in green.
  • Figure 4: Sample Efficiency. This figure plots the percentage of the benchmark dataset used in training against the character error rate, for different OCR model architectures.
  • Figure S-1: Dataset Description. Representative samples of the publications examined in this study.