Harnessing Diverse Perspectives: A Multi-Agent Framework for Enhanced Error Detection in Knowledge Graphs

TL;DR

MAKGED tackles knowledge-graph error detection by orchestrating four LLM-based agents over bidirectional subgraphs, fusing structural cues from a Graph Convolutional Network with semantic cues from an LLM into a unified representation $ abla e_concat$. The framework trains the agents to analyze head- and tail-centric subgraphs through multi-round discussions and majority voting, with a summarizer resolving ties to ensure transparent, traceable decisions. Empirical results on FB15K and WN18RR show state-of-the-art improvements in Accuracy, F1-Score, Precision, and Recall, and its industrial case studies (e.g., China Mobile) demonstrate practical value for domain-specific KGs. The work provides a scalable, interpretable approach to KG quality assurance, with released code and data for reproducibility.

Abstract

Knowledge graphs are widely used in industrial applications, making error detection crucial for ensuring the reliability of downstream applications. Existing error detection methods often fail to effectively utilize fine-grained subgraph information and rely solely on fixed graph structures, while also lacking transparency in their decision-making processes, which results in suboptimal detection performance. In this paper, we propose a novel Multi-Agent framework for Knowledge Graph Error Detection (MAKGED) that utilizes multiple large language models (LLMs) in a collaborative setting. By concatenating fine-grained, bidirectional subgraph embeddings with LLM-based query embeddings during training, our framework integrates these representations to produce four specialized agents. These agents utilize subgraph information from different dimensions to engage in multi-round discussions, thereby improving error detection accuracy and ensuring a transparent decision-making process. Extensive experiments on FB15K and WN18RR demonstrate that MAKGED outperforms state-of-the-art methods, enhancing the accuracy and robustness of KG evaluation. For specific industrial scenarios, our framework can facilitate the training of specialized agents using domain-specific knowledge graphs for error detection, which highlights the potential industrial application value of our framework. Our code and datasets are available at https://github.com/kse-ElEvEn/MAKGED.

Figures (3)

• Figure 1: Training method for bidirectional subgraph agents in MAKGED: We first construct bidirectional subgraphs for the head and tail entities of the triple, which are represented as embedding vectors using GCN. These embeddings are then concatenated with the query embeddings of the Llama2 model, resulting in four bidirectional subgraph agents to evaluate the correctness of the triples.
• Figure 2: This figure illustrates the collaborative decision-making process using multiple agents. In the "analysis" phase, the four agents independently evaluate the triple. If no consensus is reached, they proceed to the "cooperation" phase for discussion. The final decision is made either by majority rule after three rounds of discussion, or by a summarizer in case of a 2-vs-2 tie.
• Figure 3: The figure shows an example of using our framework on the China Mobile business scenario knowledge graph. We compare the evaluation results of the original Llama2 model without subgraph fine-tuning, a method without multi-agent discussion, and our complete MAKGED framework.