Retrieval augmented text-to-SQL generation for epidemiological question answering using electronic health records
Angelo Ziletti, Leonardo D'Ambrosi
TL;DR
This paper tackles answering epidemiological questions using real-world data (EHR/claims) by combining text-to-SQL generation with retrieval augmented generation (RAG) and a medical coding step to map entities to clinical ontologies. It introduces a dataset of 306 question–SQL pairs aligned to OMOP-CDM and demonstrates that RAG substantially improves both accuracy and executability over static prompting across multiple large language models. The results show GPT-4 Turbo as the strongest performer, with RAG offering consistent gains in this domain, though overall performance remains imperfect, underscoring the potential and need for cautious, supervised deployment in industry contexts.
Abstract
Electronic health records (EHR) and claims data are rich sources of real-world data that reflect patient health status and healthcare utilization. Querying these databases to answer epidemiological questions is challenging due to the intricacy of medical terminology and the need for complex SQL queries. Here, we introduce an end-to-end methodology that combines text-to-SQL generation with retrieval augmented generation (RAG) to answer epidemiological questions using EHR and claims data. We show that our approach, which integrates a medical coding step into the text-to-SQL process, significantly improves the performance over simple prompting. Our findings indicate that although current language models are not yet sufficiently accurate for unsupervised use, RAG offers a promising direction for improving their capabilities, as shown in a realistic industry setting.