Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Variance-reduced Estimation from Generative EHR Models: The SCOPE and REACH Estimators

Luke Solo, Matthew B. A. McDermott, William F. Parker, Bashar Ramadan, Michael C. Burkhart, Brett K. Beaulieu-Jones

TL;DR

This work addresses the high variance and computational burden of predicting clinical outcomes with generative EHR models by introducing two estimators, SCOPE and REACH, that exploit next-token probabilities discarded by standard Monte Carlo sampling. SCOPE sums conditional outcome probabilities across generated timelines, while REACH uses an outcome-free backbone to form a survival-process-based estimator with guaranteed lower variance. The authors prove unbiasedness for both estimators, with REACH offering a provable variance reduction over Monte Carlo and SCOPE under broad conditions. Empirical validation on MIMIC-IV via the ETHOS-ARES framework shows that SCOPE and REACH can match 100-sample Monte Carlo performance using only about 10–11 samples for hospital mortality, yielding roughly 9x inference-time reductions, and still provide favorable calibration; ICU admission gains are smaller but still present. These results enhance the practical feasibility of deploying generative EHR models in resource-constrained clinical settings and enable smoother, continuous risk trajectories for decision support.

Abstract

Generative models trained using self-supervision of tokenized electronic health record (EHR) timelines show promise for clinical outcome prediction. This is typically done using Monte Carlo simulation for future patient trajectories. However, existing approaches suffer from three key limitations: sparse estimate distributions that poorly differentiate patient risk levels, extreme computational costs, and high sampling variance. We propose two new estimators: the Sum of Conditional Outcome Probability Estimator (SCOPE) and Risk Estimation from Anticipated Conditional Hazards (REACH), that leverage next-token probability distributions discarded by standard Monte Carlo. We prove both estimators are unbiased and that REACH guarantees variance reduction over Monte Carlo sampling for any model and outcome. Empirically, on hospital mortality prediction in MIMIC-IV using the ETHOS-ARES framework, SCOPE and REACH match 100-sample Monte Carlo performance using only 10-11 samples (95% CI: [9,11]), representing a ~10x reduction in inference cost without degrading calibration. For ICU admission prediction, efficiency gains are more modest (~1.2x), which we attribute to the outcome's lower "spontaneity," a property we characterize theoretically and empirically. These methods substantially improve the feasibility of deploying generative EHR models in resource-constrained clinical settings.

Efficient Variance-reduced Estimation from Generative EHR Models: The SCOPE and REACH Estimators

TL;DR

This work addresses the high variance and computational burden of predicting clinical outcomes with generative EHR models by introducing two estimators, SCOPE and REACH, that exploit next-token probabilities discarded by standard Monte Carlo sampling. SCOPE sums conditional outcome probabilities across generated timelines, while REACH uses an outcome-free backbone to form a survival-process-based estimator with guaranteed lower variance. The authors prove unbiasedness for both estimators, with REACH offering a provable variance reduction over Monte Carlo and SCOPE under broad conditions. Empirical validation on MIMIC-IV via the ETHOS-ARES framework shows that SCOPE and REACH can match 100-sample Monte Carlo performance using only about 10–11 samples for hospital mortality, yielding roughly 9x inference-time reductions, and still provide favorable calibration; ICU admission gains are smaller but still present. These results enhance the practical feasibility of deploying generative EHR models in resource-constrained clinical settings and enable smoother, continuous risk trajectories for decision support.

Abstract

Generative models trained using self-supervision of tokenized electronic health record (EHR) timelines show promise for clinical outcome prediction. This is typically done using Monte Carlo simulation for future patient trajectories. However, existing approaches suffer from three key limitations: sparse estimate distributions that poorly differentiate patient risk levels, extreme computational costs, and high sampling variance. We propose two new estimators: the Sum of Conditional Outcome Probability Estimator (SCOPE) and Risk Estimation from Anticipated Conditional Hazards (REACH), that leverage next-token probability distributions discarded by standard Monte Carlo. We prove both estimators are unbiased and that REACH guarantees variance reduction over Monte Carlo sampling for any model and outcome. Empirically, on hospital mortality prediction in MIMIC-IV using the ETHOS-ARES framework, SCOPE and REACH match 100-sample Monte Carlo performance using only 10-11 samples (95% CI: [9,11]), representing a ~10x reduction in inference cost without degrading calibration. For ICU admission prediction, efficiency gains are more modest (~1.2x), which we attribute to the outcome's lower "spontaneity," a property we characterize theoretically and empirically. These methods substantially improve the feasibility of deploying generative EHR models in resource-constrained clinical settings.
Paper Structure (29 sections, 51 equations, 4 figures, 3 tables)

This paper contains 29 sections, 51 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction and Related Work
  2. Description of Estimators
  3. Introduction of Notation
  4. Standard Monte Carlo Generation M0
  5. Sum of Conditional Outcome Probability Estimates (SCOPE)
  6. Risk Estimation from Anticipated Conditional Hazards (REACH)
  7. Theoretical Guarantees
  8. Proofs
  9. Derivation and Unbiasedness of S (SCOPE)
  10. Characterization of the Variance of S
  11. Derivation and Unbiasedness of R (REACH)
  12. Characterization of the Variance of R
  13. Experimental Methods
  14. Markov Chain Experiments
  15. Empirical Implementation Using ETHOS-ARES
  16. ...and 14 more sections

Figures (4)

  • Figure 1: (a) Distribution of estimators vs true probability. (b) Variance of Estimator vs Probability. (c) Variance vs Spontaneity. (d) Variance vs Number of Sampled Timelines.
  • Figure 2: (a) Hospital Mortality AUC vs number of sampled timelines. (b) Hospital Mortality calibration curves (100 samples for all estimators). (c) ICU Admission AUC vs number of sampled timelines. (d) ICU Admission calibration curves (100 samples for all estimators).
  • Figure :
  • Figure :