SocraticKG: Knowledge Graph Construction via QA-Driven Fact Extraction

TL;DR

SocraticKG introduces a QA-driven framework for knowledge graph construction from unstructured text by using 5W1H-guided QA as a structured intermediate representation to unfold document semantics before triple extraction, followed by canonicalization to produce a cohesive KG. The approach yields higher factual retention and denser, more connected graphs than state-of-the-art baselines across multiple LLM backbones, effectively addressing the coverage-connectivity trade-off. Evaluations on the MINE benchmark demonstrate the value of QA-mediated semantic scaffolding for coherent, document-grounded graph construction with strong practical implications for scalable knowledge representation. While the method increases token usage and relies on the reasoning depth of underlying LLMs, it offers a principled path toward more reliable, interpretable KG extraction from long-form text.

Abstract

Constructing Knowledge Graphs (KGs) from unstructured text provides a structured framework for knowledge representation and reasoning, yet current LLM-based approaches struggle with a fundamental trade-off: factual coverage often leads to relational fragmentation, while premature consolidation causes information loss. To address this, we propose SocraticKG, an automated KG construction method that introduces question-answer pairs as a structured intermediate representation to systematically unfold document-level semantics prior to triple extraction. By employing 5W1H-guided QA expansion, SocraticKG captures contextual dependencies and implicit relational links typically lost in direct KG extraction pipelines, providing explicit grounding in the source document that helps mitigate implicit reasoning errors. Evaluation on the MINE benchmark demonstrates that our approach effectively addresses the coverage-connectivity trade-off, achieving superior factual retention while maintaining high structural cohesion even as extracted knowledge volume substantially expands. These results highlight that QA-mediated semantic scaffolding plays a critical role in structuring semantics prior to KG extraction, enabling more coherent and reliable graph construction in subsequent stages.

Figures (3)

• Figure 1: The overall architecture of the SocraticKG framework. Given unstructured text, the method first generates atomic QA pairs through 5W1H-guided questioning, then extracts triples from these QA pairs, and finally canonicalizes the triples to produce a cohesive knowledge graph.
• Figure 2: Comparison of extraction pipelines using an example output from Gemini-2.5-flash-lite. While baseline pipelines often miss the syntactic connection in complex sentences, failing to recover the causal link between bees and genetic diversity, SoKG leverages QA-driven reasoning to explicitly reconstruct the intermediate concept. As a result, SoKG successfully recovers the complete causal chain (bees $\rightarrow$ cross-pollination $\rightarrow$ genetic diversity), whereas baselines tend to simplify or fragment this relationship.
• Figure 3: Comparison of extracted graphs for the example sentence: "Volunteers provide essential services and support to vulnerable populations, such as the homeless, the elderly, and individuals with disabilities." The nested relational path implied by this text (Volunteers$\rightarrow$Vulnerable Populations$\rightarrow$ {Homeless, Elderly, Individuals with disabilities}) is emphasized to assess relational completeness. Specifically, nodes corresponding to this path are enlarged for clear visibility, connected by thick dark blue arrows to indicate the sequence of triples, while the remaining background graph elements are displayed in light blue.