Stability-Aware Prompt Optimization for Clinical Data Abstraction

TL;DR

The paper addresses the sensitivity of clinical LLMs to prompt wording and the need to treat prompt design and uncertainty jointly. It proposes a dual-objective prompt optimization loop that balances $J(P) = \lambda_{\text{perf}} F(P) + \lambda_{\text{stab}} S(P)$ by optimizing both task performance and stability across paraphrase variants. Across MedAlign applicability/correctness and MS subtype tasks with open and proprietary models, the study shows that higher accuracy does not guarantee stability, that stable predictions tend to be more confident, and that explicitly optimizing for stability reduces flip rates in most cases, sometimes with modest accuracy costs. These findings suggest stability should be incorporated into clinical AI validation and deployment to enhance reliability across teams and prompts, acting as a practical trust signal in production systems like CHARM.

Abstract

Large language models used for clinical abstraction are sensitive to prompt wording, yet most work treats prompts as fixed and studies uncertainty in isolation. We argue these should be treated jointly. Across two clinical tasks (MedAlign applicability/correctness and MS subtype abstraction) and multiple open and proprietary models, we measure prompt sensitivity via flip rates and relate it to calibration and selective prediction. We find that higher accuracy does not guarantee prompt stability, and that models can appear well-calibrated yet remain fragile to paraphrases. We propose a dual-objective prompt optimization loop that jointly targets accuracy and stability, showing that explicitly including a stability term reduces flip rates across tasks and models, sometimes at modest accuracy cost. Our results suggest prompt sensitivity should be an explicit objective when validating clinical LLM systems.

Figures (8)

• Figure 1: Stability-aware prompt optimization loop. Starting from an initial prompt, we evaluate accuracy and flip rate (measured across paraphrased variants), identify failure cases, and use an LLM to generate candidate prompts conditioned on these failures. Candidates are scored on a joint objective balancing performance and stability, and the best-scoring prompt is selected for the next iteration.
• Figure 2: Prediction margin versus flip rate for MedGemma 4B on MedAlign applicability. Stable examples (flip rate = 0) have high margins, indicating confident predictions. As flip rate increases, margins decrease and become more variable, suggesting that uncertain predictions are more prone to flipping under prompt variation.
• Figure 3: Accuracy vs flip rate for MedAlign applicability using Llama 3 70B. Each point is one candidate prompt; the y-axis is the mean flip rate across three paraphrased variants. The vertical spread at similar accuracy levels illustrates that improving accuracy alone does not guarantee prompt stability.
• Figure 4: Summary of E3 optimizer sweeps across tasks and models. Each segment shows the mean start$\rightarrow$end trajectory (3 seeds) in accuracy--flip space; endpoints include error bars. Color encodes model; line style/marker encodes objective (accuracy-only vs joint). Adding a stability term systematically shifts endpoints toward lower flip rates, often with modest accuracy trade-offs.
• Figure 5: Flip-rate vs. ProSA-style PSS on a 50-example subset (5 prompts, $k=3$ paraphrases). The strong correlation validates our anchor-based flip-rate as a practical proxy for the symmetric PSS metric.