Customizing Open Source LLMs for Quantitative Medication Attribute Extraction across Heterogeneous EHR Systems

TL;DR

Addressing the heterogeneity of MOUD prescription data across EHRs, the paper presents a framework that uses instruction-tuned open-source LLMs to translate diverse records into a fixed JSON schema for cross-site analysis. It computes a standardized metric, $MOUD\_days$, representing medication coverage using $MOUD\_days = (number\_of\_refills + 1) \times duration$ when duration is provided, or $MOUD\_days = \frac{total\_quantity}{daily\_quantity}$ when it is not. The evaluation across five rural clinics shows larger models (e.g., Qwen2.5-32B, MedGemma-27B) achieving around 93% coverage and accuracy, with per-site variability and targeted fixes for common data quirks. The approach enables privacy-preserving, scalable cross-site MOUD exposure analyses and reduces brittle ETL, while outlining limitations and avenues for extending to additional medications and models.

Abstract

Harmonizing medication data across Electronic Health Record (EHR) systems is a persistent barrier to monitoring medications for opioid use disorder (MOUD). In heterogeneous EHR systems, key prescription attributes are scattered across differently formatted fields and freetext notes. We present a practical framework that customizes open source large language models (LLMs), including Llama, Qwen, Gemma, and MedGemma, to extract a unified set of MOUD prescription attributes (prescription date, drug name, duration, total quantity, daily quantity, and refills) from heterogeneous, site specific data and compute a standardized metric of medication coverage, \emph{MOUD days}, per patient. Our pipeline processes records directly in a fixed JSON schema, followed by lightweight normalization and cross-field consistency checks. We evaluate the system on prescription level EHR data from five clinics in a national OUD study (25{,}605 records from 1{,}257 patients), using a previously annotated benchmark of 10{,}369 records (776 patients) as the ground truth. Performance is reported as coverage (share of records with a valid, matchable output) and record-level exact-match accuracy. Larger models perform best overall: Qwen2.5-32B achieves \textbf{93.4\%} coverage with \textbf{93.0\%} exact-match accuracy across clinics, and MedGemma-27B attains \textbf{93.1\%}/\textbf{92.2\%}. A brief error review highlights three common issues and fixes: imputing missing dosage fields using within-drug norms, handling monthly/weekly injectables (e.g., Vivitrol) by setting duration from the documented schedule, and adding unit checks to prevent mass units (e.g., ``250 g'') from being misread as daily counts. By removing brittle, site-specific ETL and supporting local, privacy-preserving deployment, this approach enables consistent cross-site analyses of MOUD exposure, adherence, and retention in real-world settings.

Figures (1)

• Figure 1: An overview of the framework for MOUD extraction and calculation. The system processes heterogeneous raw EHR data, uses an LLM to translate it into a unified JSON structure, calculates MOUD days, and evaluates the final output against a ground truth dataset.