KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment

TL;DR

KARMA addresses the scalability bottleneck of enriching knowledge graphs from rapidly expanding biomedical literature by employing a collaborative multi-agent LLM system. It decomposes the KG enrichment pipeline into nine specialized agents spanning ingestion, reading, summarization, extraction, schema alignment, conflict resolution, and evaluation under a central controller. The framework leverages cross-agent verification and domain adaptive prompting to mitigate hallucinations and maintain schema consistency, achieving high LLM verified correctness and reduced conflicts on PubMed corpora across Genomics Proteomics Metabolomics. Results show substantial expansion of knowledge graphs with thousands of new entities and demonstrate that backbone selection significantly influences coverage and QA coherence. The modular design supports extension to new domains and future improvements such as hybrid neuro symbolic approaches.

Abstract

Maintaining comprehensive and up-to-date knowledge graphs (KGs) is critical for modern AI systems, but manual curation struggles to scale with the rapid growth of scientific literature. This paper presents KARMA, a novel framework employing multi-agent large language models (LLMs) to automate KG enrichment through structured analysis of unstructured text. Our approach employs nine collaborative agents, spanning entity discovery, relation extraction, schema alignment, and conflict resolution that iteratively parse documents, verify extracted knowledge, and integrate it into existing graph structures while adhering to domain-specific schema. Experiments on 1,200 PubMed articles from three different domains demonstrate the effectiveness of KARMA in knowledge graph enrichment, with the identification of up to 38,230 new entities while achieving 83.1\% LLM-verified correctness and reducing conflict edges by 18.6\% through multi-layer assessments.

Figures (6)

• Figure 1: Multi-agent LLM can parse articles into new knowledge, and integrate to existing knowledge graphs through filtering.
• Figure 2: System overview of the KARMA multi-agent architecture. Each agent is an LLM-driven module tasked with specific roles such as ingestion, summarization, entity recognition, relationship extraction, conflict resolution, and final evaluation.
• Figure 3: Comparison of prompt tokens, completion tokens, and processing time across domains.
• Figure 4: Distribution of confidence, relevance, and clarity scores of extracted genomics knowledge graph triples from KARMA.
• Figure 5: Distribution of confidence, relevance, and clarity scores of extracted proteomics knowledge graph triples from KARMA.