Reading the unreadable: Creating a dataset of 19th century English newspapers using image-to-text language models

TL;DR

This work tackles the challenge of extracting readable text from 19th‑century newspapers by applying a pre‑trained image‑to‑text LM (Pixtral 12B) to the NCSE corpus, coupled with layout‑aware bounding boxes and post‑processing. It demonstrates that Pixtral substantially outperforms baselines on historical OCR tasks, achieving a median CER around 0.01 on multiple datasets, and constructs NCSE v2.0—an extensive, categorized, text‑rich resource with 1.4 million entries and 321 million words. The pipeline includes layout detection (DocLayout‑Yolo), robust bounding‑box post‑processing, and two‑tier text classification (text type and IPTC topics), enabling reliable downstream analyses like topic distributions, readability, and event tracking. The dataset and tools are released publicly to support historical and sociological research, offering a cost‑effective, scalable solution for making archival journalism readable and searchable.

Abstract

Oscar Wilde said, "The difference between literature and journalism is that journalism is unreadable, and literature is not read." Unfortunately, The digitally archived journalism of Oscar Wilde's 19th century often has no or poor quality Optical Character Recognition (OCR), reducing the accessibility of these archives and making them unreadable both figuratively and literally. This paper helps address the issue by performing OCR on "The Nineteenth Century Serials Edition" (NCSE), an 84k-page collection of 19th-century English newspapers and periodicals, using Pixtral 12B, a pre-trained image-to-text language model. The OCR capability of Pixtral was compared to 4 other OCR approaches, achieving a median character error rate of 1%, 5x lower than the next best model. The resulting NCSE v2.0 dataset features improved article identification, high-quality OCR, and text classified into four types and seventeen topics. The dataset contains 1.4 million entries, and 321 million words. Example use cases demonstrate analysis of topic similarity, readability, and event tracking. NCSE v2.0 is freely available to encourage historical and sociological research. As a result, 21st-century readers can now share Oscar Wilde's disappointment with 19th-century journalistic standards, reading the unreadable from the comfort of their own computers.

Figures (9)

• Figure 1: Multi-modal Language models can interleave text and images, allowing them to infer image based cultural references from textual cues. This is essential for effective OCR.
• Figure 2: Overall project process showing inputs and outputs at each stage. Blue boxes use the image data, green boxes only require text, or structured data
• Figure 3: The affect of post-processing bounding boxes is shown for page 'CLD-1852-06-05_page_14'. Post-processing caused box re-classification and the removal of overlaps as well as the addition of the reading order shown as black arrows.
• Figure 4: Bounding box 'CLD-1853-07-30_page_2_B0C2R7' is split into three boxes. Overlap is shown as darker shades of blue.
• Figure 5: Pre-processing the image can significantly decrease the CER of the returned text. There is a much larger decrease in the mean CER than the median, indicating that the probability of major errors in a text is substantially reduced by pre-processing