Prior-informed optimization of treatment recommendation via bandit algorithms trained on large language model-processed historical records
Saman Nessari, Ali Bozorgi-Amiri
TL;DR
The paper tackles personalized treatment decision-making by bridging unstructured EHR data and online learning. It proposes a pipeline that uses LLMs to extract structured notes, CTGAN for data augmentation, T-learner counterfactuals, and prior-informed contextual bandits for online optimization. In a stage III colon cancer case, KernelUCB with an RBF kernel achieved the best long-term rewards (~0.60–0.61 over 5,000 rounds), with XGBoost providing robust counterfactual estimates. The work demonstrates a practical, data-efficient path toward individualized medicine and highlights future directions including prospective validation and privacy-preserving data sharing.
Abstract
Current medical practice depends on standardized treatment frameworks and empirical methodologies that neglect individual patient variations, leading to suboptimal health outcomes. We develop a comprehensive system integrating Large Language Models (LLMs), Conditional Tabular Generative Adversarial Networks (CTGAN), T-learner counterfactual models, and contextual bandit approaches to provide customized, data-informed clinical recommendations. The approach utilizes LLMs to process unstructured medical narratives into structured datasets (93.2% accuracy), uses CTGANs to produce realistic synthetic patient data (55% accuracy via two-sample verification), deploys T-learners to forecast patient-specific treatment responses (84.3% accuracy), and integrates prior-informed contextual bandits to enhance online therapeutic selection by effectively balancing exploration of new possibilities with exploitation of existing knowledge. Testing on stage III colon cancer datasets revealed that our KernelUCB approach obtained 0.60-0.61 average reward scores across 5,000 rounds, exceeding other reference methods. This comprehensive system overcomes cold-start limitations in online learning environments, improves computational effectiveness, and constitutes notable progress toward individualized medicine adapted to specific patient characteristics.