GKG-LLM: A Unified Framework for Generalized Knowledge Graph Construction

TL;DR

This work addresses the inefficiency of constructing three related knowledge graph types by proposing a unified Generalized Knowledge Graph (GKG) framework and a three-stage curriculum fine-tuning method, GKG-LLM. The approach unifies KG/EKG/CKG sub-tasks into a single seq2seq paradigm and progressively injects knowledge through KG Empowerment, EKG Enhancement, and CKG Generalization stages, using LoRA+ PEFT and a large, diverse dataset (~806K training items across 15 sub-tasks in 29 datasets, plus ~140K test items). The main contributions include the first comprehensive collection of GKG sub-task data, the three-stage learning strategy, and extensive cross-type evaluation showing consistent improvements across in-domain, counter-task, and OOD data, with strong generalization demonstrated on unseen distributions. The framework reduces resource demands while enabling broad, practical applications in NLP tasks requiring structured, event-driven, and commonsense knowledge.

Abstract

The construction of Generalized Knowledge Graph (GKG), including knowledge graph, event knowledge graph and commonsense knowledge graph, is fundamental for various natural language processing tasks. Current studies typically construct these types of graph separately, overlooking holistic insights and potential unification that could be beneficial in computing resources and usage perspectives. However, a key challenge in developing a unified framework for GKG is obstacles arising from task-specific differences. In this study, we propose a unified framework for constructing generalized knowledge graphs to address this challenge. First, we collect data from 15 sub-tasks in 29 datasets across the three types of graphs, categorizing them into in-sample, counter-task, and out-of-distribution (OOD) data. Then, we propose a three-stage curriculum learning fine-tuning framework, by iteratively injecting knowledge from the three types of graphs into the Large Language Models. Extensive experiments show that our proposed model improves the construction of all three graph types across in-domain, OOD and counter-task data.

Figures (10)

• Figure 1: An illustration of several triples and graphs. The left half shows a generalized knowledge graph. The right half includes specific examples of triples from KG, EKG, CKG and demonstrates their progressive relationship.
• Figure 2: The illustration of the data distribution for all GKG sub-tasks.
• Figure 3: Three-stage curriculum learning tuning framework of GKG-LLM. The upper part represents the GKG dataset $\mathcal{D}_G$, consisting of the unified datasets. The lower part shows the three stages of GKG training: the KG empowerment stage using the KG datasets to build foundational skills, the EKG enhancement stage using the EKG datasets to enhance specific capabilities, and the CKG generalization stage using the CKG datasets and the counter task dataset to achieve generalization of the GKG-LLM capabilities. The thick arrows between the stages represent the delivery of model parameters from base model to each version of GKG-LLM.
• Figure 4: Results of different fine-tuning orders. "K-E-C" means the fine-tuning order is KG, EKG and CKG. The following sets of experiments are similar to this one.
• Figure 5: Fine-tuning with a single type of graph and performance of different intermediate version in the GKG-LLM.