Supervised Fine Tuning of Large Language Models for Domain Specific Knowledge Graph Construction:A Case Study on Hunan's Historical Celebrities

TL;DR

This work tackles the problem of extracting structured knowledge from historical texts to build a domain-specific knowledge graph for Hunan's historical celebrities in a low-resource setting. It introduces a schema-guided, instruction-tuning pipeline with LoRA-based parameter-efficient fine-tuning on open large-language systems, coupled with a multi-source data pipeline and a JSON-based output format. The study demonstrates substantial gains in biographical information extraction, with Qwen3-8B achieving the best performance (89.3866) given 100 samples and 50 training iterations, and provides a tailored evaluation framework that combines exact and semantic matching. A case study on Mao Zedong illustrates practical KG construction and visualization in Neo4j, underscoring the approach’s potential for cost-effective digital humanities workflows and cultural heritage preservation.

Abstract

Large language models and knowledge graphs offer strong potential for advancing research on historical culture by supporting the extraction, analysis, and interpretation of cultural heritage. Using Hunan's modern historical celebrities shaped by Huxiang culture as a case study, pre-trained large models can help researchers efficiently extract key information, including biographical attributes, life events, and social relationships, from textual sources and construct structured knowledge graphs. However, systematic data resources for Hunan's historical celebrities remain limited, and general-purpose models often underperform in domain knowledge extraction and structured output generation in such low-resource settings. To address these issues, this study proposes a supervised fine-tuning approach for enhancing domain-specific information extraction. First, we design a fine-grained, schema-guided instruction template tailored to the Hunan historical celebrities domain and build an instruction-tuning dataset to mitigate the lack of domain-specific training corpora. Second, we apply parameter-efficient instruction fine-tuning to four publicly available large language models - Qwen2.5-7B, Qwen3-8B, DeepSeek-R1-Distill-Qwen-7B, and Llama-3.1-8B-Instruct - and develop evaluation criteria for assessing their extraction performance. Experimental results show that all models exhibit substantial performance gains after fine-tuning. Among them, Qwen3-8B achieves the strongest results, reaching a score of 89.3866 with 100 samples and 50 training iterations. This study provides new insights into fine-tuning vertical large language models for regional historical and cultural domains and highlights their potential for cost-effective applications in cultural heritage knowledge extraction and knowledge graph construction.

Figures (9)

• Figure 1: Overall Research Framework
• Figure 2: The Workflow of Character Text Construction
• Figure 3: Single Instruction
• Figure 4: Architectural Schematic of LoRA
• Figure 5: View of the Knowledge Graph of Modern Huxiang Heroes