Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Explainable Conversational AI for Early Diagnosis with Large Language Models

Maliha Tabassum, M Shamim Kaiser

TL;DR

This work presents an explainable, interactive diagnostic chatbot built on GPT-4o and Retrieval-Augmented Generation to address diagnostic delays and limited access to specialists. By combining a RAG knowledge base of 14 diseases with adaptive questioning, symptom tracking, and chain-of-thought-inspired reasoning, the system achieves 90.3% Top-1 accuracy and 100% Top-3 accuracy, outperforming classical ML baselines. The authors implement layered explainability, real-time state updates, and evidence-linked final diagnoses to enhance clinician trust, while deploying bias and hallucination controls. The approach is particularly relevant for low-resource settings, offering accessible, transparent, and clinically actionable diagnostic support with potential integration into electronic health records and multilingual deployments.

Abstract

Healthcare systems around the world are grappling with issues like inefficient diagnostics, rising costs, and limited access to specialists. These problems often lead to delays in treatment and poor health outcomes. Most current AI and deep learning diagnostic systems are not very interactive or transparent, making them less effective in real-world, patient-centered environments. This research introduces a diagnostic chatbot powered by a Large Language Model (LLM), using GPT-4o, Retrieval-Augmented Generation, and explainable AI techniques. The chatbot engages patients in a dynamic conversation, helping to extract and normalize symptoms while prioritizing potential diagnoses through similarity matching and adaptive questioning. With Chain-of-Thought prompting, the system also offers more transparent reasoning behind its diagnoses. When tested against traditional machine learning models like Naive Bayes, Logistic Regression, SVM, Random Forest, and KNN, the LLM-based system delivered impressive results, achieving an accuracy of 90% and Top-3 accuracy of 100%. These findings offer a promising outlook for more transparent, interactive, and clinically relevant AI in healthcare.

Towards Explainable Conversational AI for Early Diagnosis with Large Language Models

TL;DR

This work presents an explainable, interactive diagnostic chatbot built on GPT-4o and Retrieval-Augmented Generation to address diagnostic delays and limited access to specialists. By combining a RAG knowledge base of 14 diseases with adaptive questioning, symptom tracking, and chain-of-thought-inspired reasoning, the system achieves 90.3% Top-1 accuracy and 100% Top-3 accuracy, outperforming classical ML baselines. The authors implement layered explainability, real-time state updates, and evidence-linked final diagnoses to enhance clinician trust, while deploying bias and hallucination controls. The approach is particularly relevant for low-resource settings, offering accessible, transparent, and clinically actionable diagnostic support with potential integration into electronic health records and multilingual deployments.

Abstract

Healthcare systems around the world are grappling with issues like inefficient diagnostics, rising costs, and limited access to specialists. These problems often lead to delays in treatment and poor health outcomes. Most current AI and deep learning diagnostic systems are not very interactive or transparent, making them less effective in real-world, patient-centered environments. This research introduces a diagnostic chatbot powered by a Large Language Model (LLM), using GPT-4o, Retrieval-Augmented Generation, and explainable AI techniques. The chatbot engages patients in a dynamic conversation, helping to extract and normalize symptoms while prioritizing potential diagnoses through similarity matching and adaptive questioning. With Chain-of-Thought prompting, the system also offers more transparent reasoning behind its diagnoses. When tested against traditional machine learning models like Naive Bayes, Logistic Regression, SVM, Random Forest, and KNN, the LLM-based system delivered impressive results, achieving an accuracy of 90% and Top-3 accuracy of 100%. These findings offer a promising outlook for more transparent, interactive, and clinically relevant AI in healthcare.

Paper Structure

This paper contains 53 sections, 15 equations, 13 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Recent ML/DL Approaches
  4. LLM-Based Approaches
  5. System Architecture and Methodology
  6. Background Tools & Technologies
  7. RAG Architecture
  8. Azure OpenAI Services and LangChain Framework
  9. Document and Data Processing
  10. System Overview
  11. Knowledge Base Construction
  12. Data Preprocessing
  13. Integration into the System
  14. Preprocessing of Input Data
  15. Document Processing and Retrieval System
  16. ...and 38 more sections

Figures (13)

  • Figure 1: Workflow of a Simple RAG based System
  • Figure 2: System Architecture
  • Figure 3: Workflow of Knowledge Base Construction
  • Figure 4: Disease Symptom Similarity Heatmap using Jaccard Similarity Coefficients
  • Figure 5: Justification for Symptom Question Selection
  • ...and 8 more figures