Jochre 3 and the Yiddish OCR corpus

TL;DR

This work tackles OCR for historical Yiddish, a language with multiple orthographies, diacritics, and mixed alphabets. It introduces Jochre 3, a neural-network–based pipeline for page-layout analysis and glyph recognition, paired with a glyph-level Yiddish OCR Corpus annotated in Alto and a Lucene-based OCR search engine. The authors report a CER of $1.5\%$ on a test corpus, significantly outperforming existing open-source Yiddish OCR tools, and provide public access to the enhanced YBC Digital Library OCR via a searchable interface. The dataset and tools pave the way for broader application to other lesser-resourced languages and future work on language-model–driven post-correction and integration with large-scale Yiddish libraries.

Abstract

We describe the construction of a publicly available Yiddish OCR Corpus, and describe and evaluate the open source OCR tool suite Jochre 3, including an Alto editor for corpus annotation, OCR software for Alto OCR layer generation, and a customizable OCR search engine. The current version of the Yiddish OCR corpus contains 658 pages, 186K tokens and 840K glyphs. The Jochre 3 OCR tool uses various fine-tuned YOLOv8 models for top-down page layout analysis, and a custom CNN network for glyph recognition. It attains a CER of 1.5% on our test corpus, far out-performing all other existing public models for Yiddish. We analyzed the full 660M word Yiddish Book Center with Jochre 3 OCR, and the new OCR is searchable through the Yiddish Book Center OCR search engine.

Figures (4)

• Figure 1: 19th century spelling example: "zeyer shtark gevundert"
• Figure 2: A sample of historic fonts in the YBC Digital Corpus
• Figure 3: Spacing for emphasis in a 1920 publication: "keyn moyre hot er nisht gehat!"
• Figure 4: OCR searching using the Jochre 3 Search Engine