Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Automatic knowledge-graph creation from historical documents: The Chilean dictatorship as a case study

Camila Díaz, Jocelyn Dunstan, Lorena Etcheverry, Antonia Fonck, Alejandro Grez, Domingo Mery, Juan Reutter, Hugo Rojas

TL;DR

This paper addresses automatically constructing knowledge graphs from historical documents about Chile's dictatorship (1973–1990). It introduces a fixed four-entity ontology and seven relation types to constrain LLM-driven extraction, forming an end-to-end text-to-graph pipeline with document splitting, fragment-level prompts, entity resolution, and post-processing. Evaluation against a domain-expert gold standard shows high precision for Individuals, with granularity-related differences primarily affecting Organizations, Locations, and Events; the study highlights the need for better evaluation metrics and annotated corpora. The approach demonstrates potential to enhance humanities research by integrating fragmented archival data into a coherent, queryable knowledge graph while acknowledging data quality and granularity as key challenges.

Abstract

We present our results regarding the automatic construction of a knowledge graph from historical documents related to the Chilean dictatorship period (1973-1990). Our approach consists on using LLMs to automatically recognize entities and relations between these entities, and also to perform resolution between these sets of values. In order to prevent hallucination, the interaction with the LLM is grounded in a simple ontology with 4 types of entities and 7 types of relations. To evaluate our architecture, we use a gold standard graph constructed using a small subset of the documents, and compare this to the graph obtained from our approach when processing the same set of documents. Results show that the automatic construction manages to recognize a good portion of all the entities in the gold standard, and that those not recognized are mostly explained by the level of granularity in which the information is structured in the graph, and not because the automatic approach misses an important entity in the graph. Looking forward, we expect this report will encourage work on other similar projects focused on enhancing research in humanities and social science, but we remark that better evaluation metrics are needed in order to accurately fine-tune these types of architectures.

Automatic knowledge-graph creation from historical documents: The Chilean dictatorship as a case study

TL;DR

This paper addresses automatically constructing knowledge graphs from historical documents about Chile's dictatorship (1973–1990). It introduces a fixed four-entity ontology and seven relation types to constrain LLM-driven extraction, forming an end-to-end text-to-graph pipeline with document splitting, fragment-level prompts, entity resolution, and post-processing. Evaluation against a domain-expert gold standard shows high precision for Individuals, with granularity-related differences primarily affecting Organizations, Locations, and Events; the study highlights the need for better evaluation metrics and annotated corpora. The approach demonstrates potential to enhance humanities research by integrating fragmented archival data into a coherent, queryable knowledge graph while acknowledging data quality and granularity as key challenges.

Abstract

We present our results regarding the automatic construction of a knowledge graph from historical documents related to the Chilean dictatorship period (1973-1990). Our approach consists on using LLMs to automatically recognize entities and relations between these entities, and also to perform resolution between these sets of values. In order to prevent hallucination, the interaction with the LLM is grounded in a simple ontology with 4 types of entities and 7 types of relations. To evaluate our architecture, we use a gold standard graph constructed using a small subset of the documents, and compare this to the graph obtained from our approach when processing the same set of documents. Results show that the automatic construction manages to recognize a good portion of all the entities in the gold standard, and that those not recognized are mostly explained by the level of granularity in which the information is structured in the graph, and not because the automatic approach misses an important entity in the graph. Looking forward, we expect this report will encourage work on other similar projects focused on enhancing research in humanities and social science, but we remark that better evaluation metrics are needed in order to accurately fine-tune these types of architectures.
Paper Structure (16 sections, 1 figure, 2 tables)

This paper contains 16 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Our approach
  4. A simple (fixed) ontology
  5. Extracting Entities and Relations
  6. The full text-to-graph pipeline
  7. Document Splitting
  8. Prompting to extract Entities and Relations
  9. Performing Entity Resolution
  10. Graph Post-Processing
  11. Evaluation
  12. A gold standard sub-graph
  13. Results
  14. Conclusions and lessons learned
  15. Appendix
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Summary of the proposed approach.