CodeKGC: Code Language Model for Generative Knowledge Graph Construction

TL;DR

CodeKGC reframes knowledge graph construction as a code-generation problem, harnessing code-language models and schema-aware prompts to preserve the structural topology of KG data. By encoding KG schemas in Python class definitions and optionally guiding generation with rationales, CodeKGC improves both zero-shot and few-shot extraction on ADE, CoNLL04, and SciERC compared to traditional text-based prompts. Extensive ablations show the value of schema-aware prompts and rationale steps, and the approach demonstrates robustness to hard cases like overlapping triples, especially with larger code LMs. The work suggests a promising direction for structured information extraction, linking programming-language structure with semantic graph construction and enabling more reliable, scalable KG population.

Abstract

Current generative knowledge graph construction approaches usually fail to capture structural knowledge by simply flattening natural language into serialized texts or a specification language. However, large generative language model trained on structured data such as code has demonstrated impressive capability in understanding natural language for structural prediction and reasoning tasks. Intuitively, we address the task of generative knowledge graph construction with code language model: given a code-format natural language input, the target is to generate triples which can be represented as code completion tasks. Specifically, we develop schema-aware prompts that effectively utilize the semantic structure within the knowledge graph. As code inherently possesses structure, such as class and function definitions, it serves as a useful model for prior semantic structural knowledge. Furthermore, we employ a rationale-enhanced generation method to boost the performance. Rationales provide intermediate steps, thereby improving knowledge extraction abilities. Experimental results indicate that the proposed approach can obtain better performance on benchmark datasets compared with baselines. Code and datasets are available in https://github.com/zjunlp/DeepKE/tree/main/example/llm.

Figures (8)

• Figure 1: Code language model for generative knowledge graph construction. The structure of code can help to understand complex structural information in natural language and we design structural code prompts as input into the language models.
• Figure 2: Overview of the proposed CodeKGC with code languages. The original natural language is converted into code formats and then fed into the code language model which is guided by a specified task prompt. We use schema-aware prompt to preserve the relations, properties, and constraints in the knowledge graph. Inspired by COT, CodeKGC also utilizes an optional rationale-enhanced module as an intermediate reasoning step in the in-context learning samples.
• Figure 3: Code generation with rationales. We try to decompose the task of knowledge graph constructions into multiple intermediate steps. By leveraging logical reasoning chains like chain-of-thought COT, CodeKGC specializes in solving complex structural patterns.
• Figure 4: Few-shot performance comparison in ADE dataset between code-based prompt (CodeKGC) and text-based prompt (Vanilla Prompt) on different models (relation strict micro F1).
• Figure 5: Impacts of prompt variation in the CoNLL04 dataset. w/o is without schema-aware prompt and rationales. Our designed schema-aware prompt and rationale-enhanced generation obviously enhance the performance of our proposed approach.