Ground Truth Generation for Multilingual Historical NLP using LLMs

TL;DR

The paper tackles the scarcity of annotated data in historical NLP by generating ground-truth via large language models for historical French and Chinese, then fine-tuning spaCy with these synthetic labels. The approach demonstrates substantial gains on period-specific tasks (POS tagging and lemmatization for French; POS and NER for Chinese) and argues for a many-models strategy where era- or genre-specific pipelines outperform universal ones. It also highlights the practical trade-offs between using LLMs and traditional methods, showing that synthetic annotations can robustly bootstrap high-quality NLP tools for under-resourced historical corpora. The work points to scalable pathways for domain adaptation in computational humanities, while acknowledging remaining challenges in tokenization and the need for transparent evaluation frameworks. Overall, synthetic ground truth emerges as a viable, cost-effective means to extend NLP capabilities to historical multilingual corpora and beyond.

Abstract

Historical and low-resource NLP remains challenging due to limited annotated data and domain mismatches with modern, web-sourced corpora. This paper outlines our work in using large language models (LLMs) to create ground-truth annotations for historical French (16th-20th centuries) and Chinese (1900-1950) texts. By leveraging LLM-generated ground truth on a subset of our corpus, we were able to fine-tune spaCy to achieve significant gains on period-specific tests for part-of-speech (POS) annotations, lemmatization, and named entity recognition (NER). Our results underscore the importance of domain-specific models and demonstrate that even relatively limited amounts of synthetic data can improve NLP tools for under-resourced corpora in computational humanities research.

Figures (1)

• Figure 1: Comparison of POS by model over time.