A Retrieval-Augmented Knowledge Mining Method with Deep Thinking LLMs for Biomedical Research and Clinical Support

TL;DR

The paper introduces BioStrataKG, a dual-layer biomedical knowledge graph, and BioCDQA, a cross-document QA dataset, to support deep, multi-hop reasoning across vast biomedical literature. It then presents IP-RAR, a retrieval-augmented reasoning framework that combines pre-retrieval reasoning, multi-level/multi-granularity retrieval, and progressive generation with self-reflection and deep thinking to produce precise, contextually grounded answers. Empirical results show IP-RAR substantially improves retrieval F1 and answer accuracy across BioCDQA, BioASQ, and MASH-QA benchmarks, with ablation analyses confirming the necessity of each component. The framework aims to aid clinicians in synthesizing evidence for personalized treatment and to accelerate biomedical research by enabling systematic analysis of advancements and gaps, while outlining future work in multimodal integration and agent-based interactions.

Abstract

Knowledge graphs and large language models (LLMs) are key tools for biomedical knowledge integration and reasoning, facilitating structured organization of scientific articles and discovery of complex semantic relationships. However, current methods face challenges: knowledge graph construction is limited by complex terminology, data heterogeneity, and rapid knowledge evolution, while LLMs show limitations in retrieval and reasoning, making it difficult to uncover cross-document associations and reasoning pathways. To address these issues, we propose a pipeline that uses LLMs to construct a biomedical knowledge graph (BioStrataKG) from large-scale articles and builds a cross-document question-answering dataset (BioCDQA) to evaluate latent knowledge retrieval and multi-hop reasoning. We then introduce Integrated and Progressive Retrieval-Augmented Reasoning (IP-RAR) to enhance retrieval accuracy and knowledge reasoning. IP-RAR maximizes information recall through Integrated Reasoning-based Retrieval and refines knowledge via Progressive Reasoning-based Generation, using self-reflection to achieve deep thinking and precise contextual understanding. Experiments show that IP-RAR improves document retrieval F1 score by 20\% and answer generation accuracy by 25\% over existing methods. This framework helps doctors efficiently integrate treatment evidence for personalized medication plans and enables researchers to analyze advancements and research gaps, accelerating scientific discovery and decision-making.

Figures (4)

• Figure 1: Overview of the proposed framework for biomedical knowledge mining. (A) Biomedical knowledge sources, such as research papers and user queries, are processed through (B) A knowledge mining pipeline that leverages a LLM alongside the IP-RAR approach, designed to generate knowledge graph and precise answers. (C) The outputs enable diverse applications, including drug synergy/antagonism, drug repurposing, precision medicine, bottleneck analysis, research planning, and knowledge transfer.
• Figure 2: Construction Pipeline and Statistical Analysis of the Dataset. (a) BioStrataKG and BioCDQA Construction Workflow Diagram. (1) Data Collection and Processing: The process begins by converting research papers from PDF to markdown (MD) format to facilitate content extraction. (2) Structured Dataset Generation for Entity-level Knowledge Graph: An LLM is used to extract entities and relationships (Entity1, Relationship, Entity2), which are then standardized to construct an entity-level knowledge graph. This graph supports downstream tasks such as drug repurposing, drug interaction analysis for comorbid conditions, and gene-disease associations. (3) Structured Dataset Generation for Document-level Knowledge Graph: Summarization is performed using an LLM to extract key aspects such as methods, datasets, and research directions. The resulting document-level knowledge graph facilitates tasks such as research strategic planning and research paper recommendations. (4) Unstructured Dataset Generation for Multiple Documents: Integration of the entity-level and document-level knowledge graphs produces a comprehensive knowledge graph. This integrated graph enables connections across multiple documents and supports downstream tasks such as content-based factual questioning, research bottleneck analysis, knowledge transfer, trend analysis, and hotspot detection. (b) Statistics of node labels and relationship types in BioStrataKG. (c) Statistics of question categories in BioCDQA.
• Figure 3: Framework of IP-RAR. (A) Integrated Reasoning-based Retrieval: Performs pre-retrieval reasoning, extracting keywords and generating a virtual answer. Then, a multi-level, multi-granularity retrieval strategy is used to retrieve relevant text chunks, which are ranked based on relevance. (B) Progressive Reasoning-based Generation: Filters out irrelevant text chunks through explanations or self-reflection, then leverages DeepSeek-R1 for deep-thinking-based reasoning on the valid text chunks, generating a precise final response.
• Figure 4: Examples of Applications in Biomedical Research and Clinical Decision Support. (a) An Example of Formulating Scientific Questions and Planning Research. (b) An Example of Drug Interaction Research for Clinical Decision Support.