Using Medical Algorithms for Task-Oriented Dialogue in LLM-Based Medical Interviews
Rui Reis, Pedro Rangel Henriques, João Ferreira-Coimbra, Eva Oliveira, Nuno F. Rodrigues
TL;DR
Efficient, accurate anamnesis in emergency departments is hampered by time pressure, cognitive load, and biases. The authors propose a task-oriented dialogue framework organized as a Directed Acyclic Graph (DAG) that uses LLMs to transform diagnostic algorithms into a corpus of questions, with cold-start initialization, dynamic expand-prune decision making, and automatic, doctor-friendly reporting. A termination criterion based on $Termination Score = (# Unique Node Labels Answered)/(# Unique Node Labels)$ guides when interviews conclude, alongside exchange limits to prevent excessive questioning. Preliminary evaluation with five physicians showed low patient workload (NASA-TLX 15.6), high usability (SUS ~86–89) and strong satisfaction (QUIS ~8.1–8.3), suggesting improved workflow efficiency and data quality. The framework demonstrates practical potential for enhancing clinical anamnesis and documentation in high-demand settings, with future work needed to address latency and generalizability.
Abstract
We developed a task-oriented dialogue framework structured as a Directed Acyclic Graph (DAG) of medical questions. The system integrates: (1) a systematic pipeline for transforming medical algorithms and guidelines into a clinical question corpus; (2) a cold-start mechanism based on hierarchical clustering to generate efficient initial questioning without prior patient information; (3) an expand-and-prune mechanism enabling adaptive branching and backtracking based on patient responses; (4) a termination logic to ensure interviews end once sufficient information is gathered; and (5) automated synthesis of doctor-friendly structured reports aligned with clinical workflows. Human-computer interaction principles guided the design of both the patient and physician applications. Preliminary evaluation involved five physicians using standardized instruments: NASA-TLX (cognitive workload), the System Usability Scale (SUS), and the Questionnaire for User Interface Satisfaction (QUIS). The patient application achieved low workload scores (NASA-TLX = 15.6), high usability (SUS = 86), and strong satisfaction (QUIS = 8.1/9), with particularly high ratings for ease of learning and interface design. The physician application yielded moderate workload (NASA-TLX = 26) and excellent usability (SUS = 88.5), with satisfaction scores of 8.3/9. Both applications demonstrated effective integration into clinical workflows, reducing cognitive demand and supporting efficient report generation. Limitations included occasional system latency and a small, non-diverse evaluation sample.