Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MEDAKA: Construction of Biomedical Knowledge Graphs Using Large Language Models

Asmita Sengupta, David Antony Selby, Sebastian Josef Vollmer, Gerrit Großmann

TL;DR

This work tackles the challenge of building scalable, real-world biomedical knowledge graphs from unstructured sources by introducing Medaka, a pipeline that combines web scraping of publicly available drug leaflets, PDF parsing, and a prompt-based LLM to extract structured subject–relation–object triples. A majority-voting scheme and an LLM-as-a-judge framework are employed to improve reliability, producing a KG with 41,142 nodes and 466,359 edges that includes clinically relevant attributes like side effects, warnings, contraindications, dosage, storage, and ingredients. The Medaka dataset is shown to achieve high agreement with human evaluation and offers broader coverage than existing resources such as DrugBank, SIDER, and FAERS. The approach is modular and adaptable to other domains, enabling applications in patient safety, drug recommendations, and beyond, with reproducible code and data released publicly.

Abstract

Knowledge graphs (KGs) are increasingly used to represent biomedical information in structured, interpretable formats. However, existing biomedical KGs often focus narrowly on molecular interactions or adverse events, overlooking the rich data found in drug leaflets. In this work, we present (1) a hackable, end-to-end pipeline to create KGs from unstructured online content using a web scraper and an LLM; and (2) a curated dataset, MEDAKA, generated by applying this method to publicly available drug leaflets. The dataset captures clinically relevant attributes such as side effects, warnings, contraindications, ingredients, dosage guidelines, storage instructions and physical characteristics. We evaluate it through manual inspection and with an LLM-as-a-Judge framework, and compare its coverage with existing biomedical KGs and databases. We expect MEDAKA to support tasks such as patient safety monitoring and drug recommendation. The pipeline can also be used for constructing KGs from unstructured texts in other domains. Code and dataset are available at https://github.com/medakakg/medaka.

MEDAKA: Construction of Biomedical Knowledge Graphs Using Large Language Models

TL;DR

This work tackles the challenge of building scalable, real-world biomedical knowledge graphs from unstructured sources by introducing Medaka, a pipeline that combines web scraping of publicly available drug leaflets, PDF parsing, and a prompt-based LLM to extract structured subject–relation–object triples. A majority-voting scheme and an LLM-as-a-judge framework are employed to improve reliability, producing a KG with 41,142 nodes and 466,359 edges that includes clinically relevant attributes like side effects, warnings, contraindications, dosage, storage, and ingredients. The Medaka dataset is shown to achieve high agreement with human evaluation and offers broader coverage than existing resources such as DrugBank, SIDER, and FAERS. The approach is modular and adaptable to other domains, enabling applications in patient safety, drug recommendations, and beyond, with reproducible code and data released publicly.

Abstract

Knowledge graphs (KGs) are increasingly used to represent biomedical information in structured, interpretable formats. However, existing biomedical KGs often focus narrowly on molecular interactions or adverse events, overlooking the rich data found in drug leaflets. In this work, we present (1) a hackable, end-to-end pipeline to create KGs from unstructured online content using a web scraper and an LLM; and (2) a curated dataset, MEDAKA, generated by applying this method to publicly available drug leaflets. The dataset captures clinically relevant attributes such as side effects, warnings, contraindications, ingredients, dosage guidelines, storage instructions and physical characteristics. We evaluate it through manual inspection and with an LLM-as-a-Judge framework, and compare its coverage with existing biomedical KGs and databases. We expect MEDAKA to support tasks such as patient safety monitoring and drug recommendation. The pipeline can also be used for constructing KGs from unstructured texts in other domains. Code and dataset are available at https://github.com/medakakg/medaka.

Paper Structure

This paper contains 20 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Proposed method
  4. Overview.
  5. Graph schema.
  6. Data collection via web scraping.
  7. PDF parsing.
  8. Information extraction using LLMs.
  9. KG construction.
  10. Dataset description
  11. Evaluation
  12. Human evaluation.
  13. LLM-as-a-judge evaluation.
  14. Coverage comparison.
  15. Related work
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Overview of the proposed pipeline for constructing the Medaka KG.
  • Figure 2: Distribution of entities (left) and relations (right) in Medaka.
  • Figure 3: Subgraphs from Medaka centered on the drugs Toltertan SR and Catasart Plus, showing connections to different biomedical entities. Only a subset of nodes is displayed for clarity.
  • Figure 4: Degree distributions of three core node types in Medaka: Drug, SideEffect, and Contraindication.
  • Figure 5: Clustering of drug nodes in Medaka based on Jaccard similarity of connected biomedical entities. Each cluster groups drugs with similar attributes, with the representative drug shown as the label.