Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models

Cristian Santini, Marieke Van Erp, Mehwish Alam

TL;DR

This work tackles multilingual historical entity linking by introducing MHEL-LLaMo, an unsupervised ensemble that combines BELA-based candidate retrieval with instruction-tuned LLMs for NIL prediction and candidate selection, guided by an adaptive threshold on retrieval confidence. By routing only hard samples to the LLM and reusing the bi-encoder for easy cases, the approach achieves strong, scalable performance across four benchmarks spanning six languages without fine-tuning. The results show substantial gains over state-of-the-art methods, with robust NIL handling and clear guidance on when to deploy LLM intervention via prompt chaining. The framework enables practical, low-resource historical EL with broad humanities applicability, while highlighting language-specific limitations and areas for future improvement.

Abstract

Despite the recent advancements in NLP with the advent of Large Language Models (LLMs), Entity Linking (EL) for historical texts remains challenging due to linguistic variation, noisy inputs, and evolving semantic conventions. Existing solutions either require substantial training data or rely on domain-specific rules that limit scalability. In this paper, we present MHEL-LLaMo (Multilingual Historical Entity Linking with Large Language MOdels), an unsupervised ensemble approach combining a Small Language Model (SLM) and an LLM. MHEL-LLaMo leverages a multilingual bi-encoder (BELA) for candidate retrieval and an instruction-tuned LLM for NIL prediction and candidate selection via prompt chaining. Our system uses SLM's confidence scores to discriminate between easy and hard samples, applying an LLM only for hard cases. This strategy reduces computational costs while preventing hallucinations on straightforward cases. We evaluate MHEL-LLaMo on four established benchmarks in six European languages (English, Finnish, French, German, Italian and Swedish) from the 19th and 20th centuries. Results demonstrate that MHEL-LLaMo outperforms state-of-the-art models without requiring fine-tuning, offering a scalable solution for low-resource historical EL. The implementation of MHEL-LLaMo is available on Github.

It's All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models

TL;DR

This work tackles multilingual historical entity linking by introducing MHEL-LLaMo, an unsupervised ensemble that combines BELA-based candidate retrieval with instruction-tuned LLMs for NIL prediction and candidate selection, guided by an adaptive threshold on retrieval confidence. By routing only hard samples to the LLM and reusing the bi-encoder for easy cases, the approach achieves strong, scalable performance across four benchmarks spanning six languages without fine-tuning. The results show substantial gains over state-of-the-art methods, with robust NIL handling and clear guidance on when to deploy LLM intervention via prompt chaining. The framework enables practical, low-resource historical EL with broad humanities applicability, while highlighting language-specific limitations and areas for future improvement.

Abstract

Despite the recent advancements in NLP with the advent of Large Language Models (LLMs), Entity Linking (EL) for historical texts remains challenging due to linguistic variation, noisy inputs, and evolving semantic conventions. Existing solutions either require substantial training data or rely on domain-specific rules that limit scalability. In this paper, we present MHEL-LLaMo (Multilingual Historical Entity Linking with Large Language MOdels), an unsupervised ensemble approach combining a Small Language Model (SLM) and an LLM. MHEL-LLaMo leverages a multilingual bi-encoder (BELA) for candidate retrieval and an instruction-tuned LLM for NIL prediction and candidate selection via prompt chaining. Our system uses SLM's confidence scores to discriminate between easy and hard samples, applying an LLM only for hard cases. This strategy reduces computational costs while preventing hallucinations on straightforward cases. We evaluate MHEL-LLaMo on four established benchmarks in six European languages (English, Finnish, French, German, Italian and Swedish) from the 19th and 20th centuries. Results demonstrate that MHEL-LLaMo outperforms state-of-the-art models without requiring fine-tuning, offering a scalable solution for low-resource historical EL. The implementation of MHEL-LLaMo is available on Github.
Paper Structure (34 sections, 1 figure, 7 tables)

This paper contains 34 sections, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Approaches for MHEL
  4. Historical IE with LLMs
  5. Methodology
  6. Candidate Retrieval
  7. Adaptive Threshold
  8. Candidate Selection
  9. Hyperparameter Configuration
  10. Experimental Setup
  11. Evaluation Datasets
  12. Architecture Configuration
  13. Baseline systems
  14. Results
  15. HIPE-2020 Performance:
  16. ...and 19 more sections

Figures (1)

  • Figure 1: Overview of the MHEL-LLaMo architecture. The system combines BELA's bi-encoder for candidate retrieval, a KB consisting of a Faiss Index, and a lookup table for returning similar candidates from Wikidata with metadata, an adaptive threshold to filter easy samples and an instruction-tuned LLM for NIL prediction and candidate selection for hard samples.