Harmonizing Human Insights and AI Precision: Hand in Hand for Advancing Knowledge Graph Task

TL;DR

Knowledge graph completion via link prediction often relies on AI-dominated KGE models that lack human-level conceptual reasoning. KG-HAIT introduces a human-AI teaming framework where a human-designed dynamic programming process produces human insight features (HIF) that capture local subgraph structure and semantic similarity, which are then integrated into KGE training through dimensionality alignment and HIF-relation construction. Empirical results across TransE, TransH, and TransR on FB15k-237, WN18RR, and LastFM-9 show substantial improvements (e.g., average MR reduction of $42.8\%$, large gains in H@1) and faster convergence, demonstrating the value of combining human intuition with AI modeling for KG analysis. The findings highlight the potential of HAIT to enhance interpretability and efficiency in KG tasks and open avenues for broader human-in-the-loop strategies in knowledge graphs.

Abstract

Knowledge graph embedding (KGE) has caught significant interest for its effectiveness in knowledge graph completion (KGC), specifically link prediction (LP), with recent KGE models cracking the LP benchmarks. Despite the rapidly growing literature, insufficient attention has been paid to the cooperation between humans and AI on KG. However, humans' capability to analyze graphs conceptually may further improve the efficacy of KGE models with semantic information. To this effect, we carefully designed a human-AI team (HAIT) system dubbed KG-HAIT, which harnesses the human insights on KG by leveraging fully human-designed ad-hoc dynamic programming (DP) on KG to produce human insightful feature (HIF) vectors that capture the subgraph structural feature and semantic similarities. By integrating HIF vectors into the training of KGE models, notable improvements are observed across various benchmarks and metrics, accompanied by accelerated model convergence. Our results underscore the effectiveness of human-designed DP in the task of LP, emphasizing the pivotal role of collaboration between humans and AI on KG. We open avenues for further exploration and innovation through KG-HAIT, paving the way towards more effective and insightful KG analysis techniques.

Figures (6)

• Figure 1: Subgraph of a knowledge graph with actual relations between entities (solid lines) and inferred relations (dashed lines). Entities with the same type are presented with the same color.
• Figure 2: The workflow of how human insightful feature is obtained where it follows the order 1. HIF-entity construction, 2. dimensionality sequeezing, and 3. HIF-relation construction
• Figure 3: An example of how HIF-entity is calculated. The DP result after 3 iterations $\mathbf w^{(3)}_u$ is obtained by applying general addition on $\mathbf p(P_1)$, $\mathbf p(P_2)$, $\cdots$, $\mathbf p(P_8)$. Additionaly, $\mathbf p(P_k)$, $k \in [1, 8] \cap \mathbb Z$ is defined as the general product of all the entities and relations included in that path.
• Figure 4: The confusion matrices of cosine similarity between HIF-entity for 2 types of entities representing country/region and educational institution
• Figure 5: H@10 and MR for every 25 epochs (TransE w/o HIF and TransE w/ HIF on LastFM-9)