Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI

TL;DR

This paper addresses the critical issue of hallucinations in healthcare-focused LLMs, outlining a framework to quantify, validate, and mitigate these errors to enable trustworthy AI deployment. It surveys sources of hallucinations, reviews both human and automatic evaluation methods, and details a suite of mitigation strategies including HITL, algorithmic corrections, fine-tuning, prompt engineering, adversarial training, input validation, memory augmentation, and benchmark audits. The work emphasizes that responsible AI—through transparency, safety, and governance—is essential for real-world healthcare adoption and highlights regulatory considerations and practical guardrails as central to future progress. Overall, the paper provides a structured blueprint for reducing hallucinations and improving reliability in high-stakes medical applications of LLMs.

Abstract

Large language models have proliferated across multiple domains in as short period of time. There is however hesitation in the medical and healthcare domain towards their adoption because of issues like factuality, coherence, and hallucinations. Give the high stakes nature of healthcare, many researchers have even cautioned against its usage until these issues are resolved. The key to the implementation and deployment of LLMs in healthcare is to make these models trustworthy, transparent (as much possible) and explainable. In this paper we describe the key elements in creating reliable, trustworthy, and unbiased models as a necessary condition for their adoption in healthcare. Specifically we focus on the quantification, validation, and mitigation of hallucinations in the context in healthcare. Lastly, we discuss how the future of LLMs in healthcare may look like.