Zero-shot and Few-shot Generation Strategies for Artificial Clinical Records
Erlend Frayling, Jake Lever, Graham McDonald
TL;DR
The paper tackles privacy barriers in clinical research by generating synthetic electronic health records (EHRs) without using sensitive data for training. It evaluates zero-shot and few-shot generation of History of Present Illness narratives from Chief Complaint text using Llama 2, comparing prompting strategies against fine-tuned baselines on MIMIC-IV CC-HPI data. A novel Chain-of-Thought prompting approach guides the model through intermediate attributes before HPI generation, boosting zero-shot performance to be competitive with a fine-tuned GPT-2. Results show CoT prompting substantially enhances zero-shot generation, with fine-tuned Llama 2 (QLoRA) delivering the best overall Rouge scores, highlighting a path toward privacy-preserving, data-efficient clinical text generation. The work demonstrates practical potential for reducing reliance on real patient data in clinical research while maintaining data quality for downstream analysis.
Abstract
The challenge of accessing historical patient data for clinical research, while adhering to privacy regulations, is a significant obstacle in medical science. An innovative approach to circumvent this issue involves utilising synthetic medical records that mirror real patient data without compromising individual privacy. The creation of these synthetic datasets, particularly without using actual patient data to train Large Language Models (LLMs), presents a novel solution as gaining access to sensitive patient information to train models is also a challenge. This study assesses the capability of the Llama 2 LLM to create synthetic medical records that accurately reflect real patient information, employing zero-shot and few-shot prompting strategies for comparison against fine-tuned methodologies that do require sensitive patient data during training. We focus on generating synthetic narratives for the History of Present Illness section, utilising data from the MIMIC-IV dataset for comparison. In this work introduce a novel prompting technique that leverages a chain-of-thought approach, enhancing the model's ability to generate more accurate and contextually relevant medical narratives without prior fine-tuning. Our findings suggest that this chain-of-thought prompted approach allows the zero-shot model to achieve results on par with those of fine-tuned models, based on Rouge metrics evaluation.