RELEAP: Reinforcement-Enhanced Label-Efficient Active Phenotyping for Electronic Health Records

TL;DR

Noisy proxy labels in EHR phenotyping hinder reliable risk prediction. RELEAP introduces a reinforcement-learning–driven active-phenotyping framework that directly uses downstream performance as feedback to refine phenotypes and select samples under labeling budgets. In a DUHS lung cancer risk study, RELEAP outperformed noisy-label baselines and single-strategy AL, approaching oracle performance in both discrimination ($AUC$) and calibration ($MSE$) for logistic and time-to-event models, with smoother gains and notable improvements in subgroups. This work demonstrates a scalable, adaptive approach to label-efficient EHR phenotyping that improves downstream risk prediction and could generalize to other phenotyping tasks. The method leverages $S^{\ast}$ vs $S_{\text{true}}$ distinctions and combines multiple AL strategies via PPO to optimize information gain under budget constraints.

Abstract

Objective: Electronic health record (EHR) phenotyping often relies on noisy proxy labels, which undermine the reliability of downstream risk prediction. Active learning can reduce annotation costs, but most rely on fixed heuristics and do not ensure that phenotype refinement improves prediction performance. Our goal was to develop a framework that directly uses downstream prediction performance as feedback to guide phenotype correction and sample selection under constrained labeling budgets. Materials and Methods: We propose Reinforcement-Enhanced Label-Efficient Active Phenotyping (RELEAP), a reinforcement learning-based active learning framework. RELEAP adaptively integrates multiple querying strategies and, unlike prior methods, updates its policy based on feedback from downstream models. We evaluated RELEAP on a de-identified Duke University Health System (DUHS) cohort (2014-2024) for incident lung cancer risk prediction, using logistic regression and penalized Cox survival models. Performance was benchmarked against noisy-label baselines and single-strategy active learning. Results: RELEAP consistently outperformed all baselines. Logistic AUC increased from 0.774 to 0.805 and survival C-index from 0.718 to 0.752. Using downstream performance as feedback, RELEAP produced smoother and more stable gains than heuristic methods under the same labeling budget. Discussion: By linking phenotype refinement to prediction outcomes, RELEAP learns which samples most improve downstream discrimination and calibration, offering a more principled alternative to fixed active learning rules. Conclusion: RELEAP optimizes phenotype correction through downstream feedback, offering a scalable, label-efficient paradigm that reduces manual chart review and enhances the reliability of EHR-based risk prediction.

Figures (5)

• Figure 1: RELEAP workflow. Patient-level inputs include clinical covariates for constructing proxy phenotypes, additional structured predictors for risk models, and a limited pool of costly reference labels. The reinforcement learning-active learning (RL-AL) process adaptively selects patients to correct noisy proxy phenotypes into refined computable phenotypes, guided by a policy trained with proximal policy optimization (PPO). The state vector summarizes short-term downstream performance trends, labeling budget, and strategy-level statistics. The policy outputs weights across active learning strategies (uncertainty, diversity, and query-by-committee, QBC), which drive the action of selecting patients for relabeling. Corrected phenotypes are then combined with additional structured predictors to train downstream risk prediction models. Model performance on validation data (e.g., area under the receiver operating characteristic curve [AUC] or concordance index [C-index]) is logged as the final outcome and also fed back as the reward signal, closing the loop between downstream prediction and the RL-AL process.
• Figure 2: Coverage of smoking-related ICD codes (presence of $\geq 1$ smoking-related ICD per patient).
• Figure 3: Validation Metrics vs. AL Iteration in the Logistic Mode (mean $\pm$ 95% CI over ten replications).
• Figure 4: C-index vs. AL Iteration in the Survival (Cox) Mode (mean $\pm$ 95% CI over ten replications).
• Figure 5: Validation Metrics vs. AL Iteration in the Logistic Mode (mean $\pm$ 95% CI over ten replications) within male and female subgroups.