Named Entity Recognition in Historical Italian: The Case of Giacomo Leopardi's Zibaldone

TL;DR

This paper tackles NER in historical Italian, using Leopardi’s Zibaldone to create a publicly available benchmark for person, location, and bibliographic references. It constructs DigitalZibaldone-derived annotations (2,899 mentions across 260 evaluation notes and 688 training notes) linked to Wikidata or VIAF and compares instruction-tuned LLaMa3.1 prompts with a domain-specific GliNER model fine-tuned on the data. The results show domain-specific fine-tuning substantially outperforms in-context LLM methods, with micro-F1 of 68.98% (exact) and 75.64% (fuzzy), and the person class proving most reliable while bibliographic works remain challenging. The study highlights the necessity of domain adaptation and human-in-the-loop curation for reliable historical NER and outlines future work on entity linking and knowledge-base-informed disambiguation.

Abstract

The increased digitization of world's textual heritage poses significant challenges for both computer science and literary studies. Overall, there is an urgent need of computational techniques able to adapt to the challenges of historical texts, such as orthographic and spelling variations, fragmentary structure and digitization errors. The rise of large language models (LLMs) has revolutionized natural language processing, suggesting promising applications for Named Entity Recognition (NER) on historical documents. In spite of this, no thorough evaluation has been proposed for Italian texts. This research tries to fill the gap by proposing a new challenging dataset for entity extraction based on a corpus of 19th century scholarly notes, i.e. Giacomo Leopardi's Zibaldone (1898), containing 2,899 references to people, locations and literary works. This dataset was used to carry out reproducible experiments with both domain-specific BERT-based models and state-of-the-art LLMs such as LLaMa3.1. Results show that instruction-tuned models encounter multiple difficulties handling historical humanistic texts, while fine-tuned NER models offer more robust performance even with challenging entity types such as bibliographic references.