Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data

Rumeng Li, Xun Wang, Hong Yu

TL;DR

This work addresses the challenge of detecting Alzheimer's disease signs in electronic health records under data scarcity and privacy constraints. It introduces a nine-category taxonomy of AD signs and two LLM-driven data augmentation directions: data-to-label (annotation) and label-to-data (generation). The authors create gold, silver, and bronze datasets and show that incorporating silver and bronze data generally improves classifier performance, with particular gains for minority categories; the label-to-data approach also yields privacy-preserving synthetic data with acceptable quality. The findings demonstrate that expert-guided LLM data generation can augment complex clinical NLP tasks, while highlighting quality controls and ethical considerations essential for responsible AI in healthcare.

Abstract

Large language models (LLMs) can generate natural language texts for various domains and tasks, but their potential for clinical text mining, a domain with scarce, sensitive, and imbalanced medical data, is underexplored. We investigate whether LLMs can augment clinical data for detecting Alzheimer's Disease (AD)-related signs and symptoms from electronic health records (EHRs), a challenging task that requires high expertise. We create a novel pragmatic taxonomy for AD sign and symptom progression based on expert knowledge, which guides LLMs to generate synthetic data following two different directions: "data-to-label", which labels sentences from a public EHR collection with AD-related signs and symptoms; and "label-to-data", which generates sentences with AD-related signs and symptoms based on the label definition. We train a system to detect AD-related signs and symptoms from EHRs, using three datasets: (1) a gold dataset annotated by human experts on longitudinal EHRs of AD patients; (2) a silver dataset created by the data-to-label method; and (3) a bronze dataset created by the label-to-data method. We find that using the silver and bronze datasets improves the system performance, outperforming the system using only the gold dataset. This shows that LLMs can generate synthetic clinical data for a complex task by incorporating expert knowledge, and our label-to-data method can produce datasets that are free of sensitive information, while maintaining acceptable quality.

Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data

TL;DR

This work addresses the challenge of detecting Alzheimer's disease signs in electronic health records under data scarcity and privacy constraints. It introduces a nine-category taxonomy of AD signs and two LLM-driven data augmentation directions: data-to-label (annotation) and label-to-data (generation). The authors create gold, silver, and bronze datasets and show that incorporating silver and bronze data generally improves classifier performance, with particular gains for minority categories; the label-to-data approach also yields privacy-preserving synthetic data with acceptable quality. The findings demonstrate that expert-guided LLM data generation can augment complex clinical NLP tasks, while highlighting quality controls and ethical considerations essential for responsible AI in healthcare.

Abstract

Large language models (LLMs) can generate natural language texts for various domains and tasks, but their potential for clinical text mining, a domain with scarce, sensitive, and imbalanced medical data, is underexplored. We investigate whether LLMs can augment clinical data for detecting Alzheimer's Disease (AD)-related signs and symptoms from electronic health records (EHRs), a challenging task that requires high expertise. We create a novel pragmatic taxonomy for AD sign and symptom progression based on expert knowledge, which guides LLMs to generate synthetic data following two different directions: "data-to-label", which labels sentences from a public EHR collection with AD-related signs and symptoms; and "label-to-data", which generates sentences with AD-related signs and symptoms based on the label definition. We train a system to detect AD-related signs and symptoms from EHRs, using three datasets: (1) a gold dataset annotated by human experts on longitudinal EHRs of AD patients; (2) a silver dataset created by the data-to-label method; and (3) a bronze dataset created by the label-to-data method. We find that using the silver and bronze datasets improves the system performance, outperforming the system using only the gold dataset. This shows that LLMs can generate synthetic clinical data for a complex task by incorporating expert knowledge, and our label-to-data method can produce datasets that are free of sensitive information, while maintaining acceptable quality.
Paper Structure (24 sections, 1 figure, 4 tables)

This paper contains 24 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models
  4. Clinical Text Mining and synthetic data generation
  5. Alzheimer's disease signs and symptoms detection
  6. Methodology
  7. Task overview
  8. Datasets
  9. Gold data
  10. Silver data
  11. Bronze data
  12. Experiments
  13. Classifiers
  14. Data Combinations
  15. Results and Discussion
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Illustration of the text and annotation generated by the GPT-4 based on the annotation guideline. (Class names are shown here for better readability.)