InvertiTune: High-Quality Data Synthesis for Cost-Effective Single-Shot Text-to-Knowledge Graph Generation

TL;DR

InvertiTune tackling Text2KG reduces reliance on costly iterative prompting by generating high-quality (text, KG) training pairs via a controlled subgraph extraction from Wikidata and LLM-based text synthesis. Fine-tuning a lightweight LLM (Qwen 2.5-1.5B Instruct) on CE12k enables single-pass KG generation, outperforming larger non-fine-tuned models and existing Text2KG baselines. The approach delivers strong cross-dataset generalization, with CrossEval-1200 showing improved transfer to unseen distributions, and demonstrates that dataset quality and targeted filtering drive performance even with modest data amounts. Overall, the work highlights the value of realistic, high-quality training data for efficient, scalable Text2KG systems and provides a public dataset to foster further research.

Abstract

Large Language Models (LLMs) have revolutionized the ability to understand and generate text, enabling significant progress in automatic knowledge graph construction from text (Text2KG). Many Text2KG methods, however, rely on iterative LLM prompting, making them computationally expensive and prone to overlooking complex relations distributed throughout the text. To address these limitations, we propose InvertiTune, a framework that combines a controlled data generation pipeline with supervised fine-tuning (SFT). Within this framework, the data-generation pipeline systematically extracts subgraphs from large knowledge bases, applies noise filtering, and leverages LLMs to generate corresponding natural text descriptions, a task more aligned with LLM capabilities than direct KG generation from text. This pipeline enables generating datasets composed of longer texts paired with larger KGs that better reflect real-world scenarios compared to existing benchmarks, thus supporting effective SFT of lightweight models for single-shot KG construction. Experimental results on CE12k, a dataset generated using the introduced pipeline, show that InvertiTune outperforms larger non-fine-tuned LLMs as well as state-of-the-art Text2KG approaches, while also demonstrating stronger cross-dataset generalization on CrossEval-1200, a test set created from three established benchmark datasets and CE12k. These findings highlight the importance of realistic, high-quality training data for advancing efficient and high-performing Text2KG systems.

Figures (11)

• Figure 1: An overview of our InvertiTune framework, including the data generation pipeline and the training phase (SFT for the Text2KG task).
• Figure 2: Convergence of token count distributions between predicted and ground-truth knowledge graphs across dataset sizes ($N=2000$–$12000$). The left plot shows the Wasserstein distance, while the right panels present kernel density estimates, indicating closer alignment as $N$ increases.
• Figure 3: Convergence of triple count distributions between predicted and ground-truth knowledge graphs across dataset sizes ($N=2000$–$12000$). The left plot shows the Wasserstein distance, while the right panels present kernel density estimates, with convergence becoming more evident as $N$ increases.
• Figure 4: BERTScore F1 improvement of the InvertiTune model over the Qwen2.5-1.5B Instruct model as a function of sample size. Values are derived from the statistics reported in Table \ref{['tab:sft_comparison']}.
• Figure 5: Performance of InvertiTune under SFT across varying dataset sizes, evaluated using KG-based metrics.