Multimodal LLMs for Historical Dataset Construction from Archival Image Scans: German Patents (1877-1918)

TL;DR

This work shows how multimodal LLMs can transform historical dataset construction by converting archival image scans of German patent registers (1877–1918) into a large, structured patent database. The authors implement a two-stage pipeline powered by Gemini-2.5-Pro and Gemini-2.5-Flash-Lite to extract 306,070 patent entries from 9,562 pages, and they rigorously benchmark against a student-constructed and a perfect dataset to assess transcription and variable extraction quality. They report substantial efficiency gains (roughly 795x faster and 205x cheaper) and provide an open-source data pipeline and dataset to enable replication and adaptation for other collections. The study also discusses the economic and methodological implications for economic history, emphasizing the need for careful validation to mitigate hallucinations and bias in AI-assisted data construction.

Abstract

We leverage multimodal large language models (LLMs) to construct a dataset of 306,070 German patents (1877-1918) from 9,562 archival image scans using our LLM-based pipeline powered by Gemini-2.5-Pro and Gemini-2.5-Flash-Lite. Our benchmarking exercise provides tentative evidence that multimodal LLMs can create higher quality datasets than our research assistants, while also being more than 795 times faster and 205 times cheaper in constructing the patent dataset from our image corpus. About 20 to 50 patent entries are embedded on each page, arranged in a double-column format and printed in Gothic and Roman fonts. The font and layout complexity of our primary source material suggests to us that multimodal LLMs are a paradigm shift in how datasets are constructed in economic history. We open-source our benchmarking and patent datasets as well as our LLM-based data pipeline, which can be easily adapted to other image corpora using LLM-assisted coding tools, lowering the barriers for less technical researchers. Finally, we explain the economics of deploying LLMs for historical dataset construction and conclude by speculating on the potential implications for the field of economic history.

Figures (11)

• Figure 1: THE PRIMARY SOURCE
• Figure 2: A PATENT ENTRY
• Figure 3: OUR LLM-BASED DATA PIPELINE
• Figure 4: INTERFACE FOR ACCELERATED MANUAL DATA CLEANING
• Figure 5: CHARACTER ERROR RATE BY YEARLY VOLUME: RESEARCH ASSISTANTS VS LLM