Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MedSyn: LLM-based Synthetic Medical Text Generation Framework

Gleb Kumichev, Pavel Blinov, Yulia Kuzkina, Vasily Goncharov, Galina Zubkova, Nikolai Zenovkin, Aleksei Goncharov, Andrey Savchenko

TL;DR

MedSyn is introduced, a novel medical text generation framework that integrates large language models with a Medical Knowledge Graph (MKG) to sample prior medical information for the prompt and generate synthetic clinical notes with GPT-4 and fine-tuned LLaMA models.

Abstract

Generating synthetic text addresses the challenge of data availability in privacy-sensitive domains such as healthcare. This study explores the applicability of synthetic data in real-world medical settings. We introduce MedSyn, a novel medical text generation framework that integrates large language models with a Medical Knowledge Graph (MKG). We use MKG to sample prior medical information for the prompt and generate synthetic clinical notes with GPT-4 and fine-tuned LLaMA models. We assess the benefit of synthetic data through application in the ICD code prediction task. Our research indicates that synthetic data can increase the classification accuracy of vital and challenging codes by up to 17.8% compared to settings without synthetic data. Furthermore, to provide new data for further research in the healthcare domain, we present the largest open-source synthetic dataset of clinical notes for the Russian language, comprising over 41k samples covering 219 ICD-10 codes.

MedSyn: LLM-based Synthetic Medical Text Generation Framework

TL;DR

MedSyn is introduced, a novel medical text generation framework that integrates large language models with a Medical Knowledge Graph (MKG) to sample prior medical information for the prompt and generate synthetic clinical notes with GPT-4 and fine-tuned LLaMA models.

Abstract

Generating synthetic text addresses the challenge of data availability in privacy-sensitive domains such as healthcare. This study explores the applicability of synthetic data in real-world medical settings. We introduce MedSyn, a novel medical text generation framework that integrates large language models with a Medical Knowledge Graph (MKG). We use MKG to sample prior medical information for the prompt and generate synthetic clinical notes with GPT-4 and fine-tuned LLaMA models. We assess the benefit of synthetic data through application in the ICD code prediction task. Our research indicates that synthetic data can increase the classification accuracy of vital and challenging codes by up to 17.8% compared to settings without synthetic data. Furthermore, to provide new data for further research in the healthcare domain, we present the largest open-source synthetic dataset of clinical notes for the Russian language, comprising over 41k samples covering 219 ICD-10 codes.
Paper Structure (23 sections, 2 equations, 8 figures, 4 tables)

This paper contains 23 sections, 2 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Medical Knowledge Graphs
  4. LLMs in Medical Domain
  5. Method
  6. Medical Knowledge Graph
  7. Instruction-Following Dataset
  8. Fine-Tuning
  9. Generation Task
  10. Symptoms Sampling
  11. Synthetic Dataset
  12. Experiments
  13. Datasets and Tasks
  14. Models
  15. Evaluation
  16. ...and 8 more sections

Figures (8)

  • Figure 1: Examples of real clinical notes from RuMedPrime dataset rumedprime (translated to English).
  • Figure 2: The clinical notes generation pipeline implemented in MedSyn framework. Relevant symptoms from MKG and clinical note examples corresponding to the ICD code are compiled into a prompt and used as input for LLM inference.
  • Figure 3: Examples of k-hop reasoning question on MKG. Di - Disease, Dr - Drug, S - Symptoms.
  • Figure 4: The structure of the instruction-following dataset. Leaves represent data sources and the percentage of data relative to the parent category.
  • Figure 5: BERT-scores for example and symptoms usage.
  • ...and 3 more figures