BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science

TL;DR

BioKGBench presents a novel benchmark for evaluating AI scientists in biomedicine by decomposing literature understanding into grounded knowledge graph querying (KGQA) and literature-based claim verification (SCV), and introducing KGCheck to test end-to-end correctness using both structured and unstructured data. It builds a large, multi-part dataset from the Clinical Knowledge Graph and biomedical literature, and demonstrates the challenges current AI agents face by proposing BKGAgent, a simple three-agent baseline that leverages KG grounding and external evidence retrieval. The work provides a comprehensive experimental analysis across multiple LLMs, highlighting performance gaps between API-based and open-source models, the impact of retrieval scope, and the critical role of system prompts and leadership in multi-agent workflows. Overall, BioKGBench offers a dynamic, tool-augmented framework with practical implications for developing and evaluating AI scientists in biomedical research.

Abstract

Pursuing artificial intelligence for biomedical science, a.k.a. AI Scientist, draws increasing attention, where one common approach is to build a copilot agent driven by Large Language Models (LLMs). However, to evaluate such systems, people either rely on direct Question-Answering (QA) to the LLM itself, or in a biomedical experimental manner. How to precisely benchmark biomedical agents from an AI Scientist perspective remains largely unexplored. To this end, we draw inspiration from one most important abilities of scientists, understanding the literature, and introduce BioKGBench. In contrast to traditional evaluation benchmark that only focuses on factual QA, where the LLMs are known to have hallucination issues, we first disentangle "Understanding Literature" into two atomic abilities, i) "Understanding" the unstructured text from research papers by performing scientific claim verification, and ii) Ability to interact with structured Knowledge-Graph Question-Answering (KGQA) as a form of "Literature" grounding. We then formulate a novel agent task, dubbed KGCheck, using KGQA and domain-based Retrieval-Augmented Generation (RAG) to identify the factual errors of existing large-scale knowledge graph databases. We collect over two thousand data for two atomic tasks and 225 high-quality annotated data for the agent task. Surprisingly, we discover that state-of-the-art agents, both daily scenarios and biomedical ones, have either failed or inferior performance on our benchmark. We then introduce a simple yet effective baseline, dubbed BKGAgent. On the widely used popular knowledge graph, we discover over 90 factual errors which provide scenarios for agents to make discoveries and demonstrate the effectiveness of our approach. The code and data are available at https://github.com/westlake-autolab/BioKGBench.

Figures (37)

• Figure 1: (Left) Previous benchmarks for domain-specific AI Agents either focus on the low-level tasks like question answering or are embedded in a complicated pipeline as a scientist copilot. (Right) We close the gap by constructing a knowledge graph checking task that consists of two atomic sub-tasks: Knowledge Graph Question Answering (KGQA) and Scientific Claim Verification (SCV), to provide a better evaluation of AI Agents in biomedical science domain.
• Figure 2: The sub-graph of the Clinical Knowledge Graph (CKG) retains 12 types of nodes and 18 kinds of relationships.
• Figure 3: Framework of our BKGAgent.
• Figure 4: Llama-3-70B-Instruct's performance in RAG across different scopes of literature.
• Figure 5: Error analysis. Here, we show a failure case due to a leader's various mistakes: the hallucination of the leader misleading the later task or using the wrong process, the leader producing unnecessary repeated tasks and misunderstanding leads to the wrong process.