DALL-M: Context-Aware Clinical Data Augmentation with LLMs

TL;DR

DALL-M presents a context-aware data augmentation framework for clinical tabular data by grounding LLM-driven feature generation in domain knowledge retrieved from Radiopaedia and Wikipedia. The three-phase pipeline—clinical context extraction and storage, expert query generation, and context-aware feature augmentation—uses retrieval-augmented generation to produce both contextual values for existing features and entirely new clinically relevant features, anchored to trusted sources. Empirical evaluation on 799 MIMIC-IV cases with REFLACX-labeled lesions shows substantial improvements in predictive performance (e.g., a 16.5% increase in F1 and a 25% boost in Precision/Recall for certain models) and expansion from 9 to 91 features with expert validation. The work demonstrates that context-aware, knowledge-grounded augmentation can enhance AI-driven medical diagnostics while preserving clinical validity, offering a scalable approach for data-scarce healthcare settings.

Abstract

X-ray images are vital in medical diagnostics, but their effectiveness is limited without clinical context. Radiologists often find chest X-rays insufficient for diagnosing underlying diseases, necessitating the integration of structured clinical features with radiology reports. To address this, we introduce DALL-M, a novel framework that enhances clinical datasets by generating contextual synthetic data. DALL-M augments structured patient data, including vital signs (e.g., heart rate, oxygen saturation), radiology findings (e.g., lesion presence), and demographic factors. It integrates this tabular data with contextual knowledge extracted from radiology reports and domain-specific resources (e.g., Radiopaedia, Wikipedia), ensuring clinical consistency and reliability. DALL-M follows a three-phase process: (i) clinical context storage, (ii) expert query generation, and (iii) context-aware feature augmentation. Using large language models (LLMs), it generates both contextual synthetic values for existing clinical features and entirely new, clinically relevant features. Applied to 799 cases from the MIMIC-IV dataset, DALL-M expanded the original 9 clinical features to 91. Empirical validation with machine learning models (including Decision Trees, Random Forests, XGBoost, and TabNET) demonstrated a 16.5% improvement in F1 score and a 25% increase in Precision and Recall. DALL-M bridges an important gap in clinical data augmentation by preserving data integrity while enhancing predictive modeling in healthcare. Our results show that integrating LLM-generated synthetic features significantly improves model performance, making DALL-M a scalable and practical approach for AI-driven medical diagnostics.

Figures (7)

• Figure 1: Overview of the DALL-M framework for generating clinically relevant features using Large Language Models. The process consists of three main phases: (I) Clinical Context Extraction and Storage, where patient-specific contexts and clinical relationships are captured and stored; (II) Expert Input Queries and Prompt Generation, where medical experts provide contextual queries to guide the LLM; and (III) Context-Aware Clinical Feature Augmentation, where new features are generated in alignment with the clinical context, ensuring that the augmented data remains consistent and clinically valid.
• Figure 2: Example of a prompt used in Experiment I for generating feature values for Oxygen Saturation (%).
• Figure 3: Example of feature importance distribution for enlarged cardiac silhouette using Decision Trees, Random Forests and XGBoost.
• Figure 4: Example of feature importance distribution for enlarged cardiac silhouette using TabNet
• Figure 5: Correlation of Machine Learning Models with Radiologists' Clinical Scores for High and Low Relevance Features