MedSpeak: A Knowledge Graph-Aided ASR Error Correction Framework for Spoken Medical QA

TL;DR

This work tackles high-stakes spoken medical QA where ASR errors on medical terms degrade downstream reasoning. MedSpeak integrates a UMLS-derived medical knowledge graph with phonetic cues, via a two-layer KG and a fine-tuned LLM, to jointly correct transcripts and select answers, with the training objective given by $\mathcal{L}(\theta) = - \sum_{i=1}^{|y|} \log P_{\theta}(y_i \mid x, y_{<i})$. Empirical results across 47 hours of synthesized data from MMLU-Medical, MedQA, and MedMCQA show MedSpeak achieves an overall QA accuracy of 93.4% and a WER of 29.9%, outperforming zero-shot and pure fine-tuning baselines. The work demonstrates that incorporating phonetic knowledge with semantic KG context in an end-to-end LLM framework yields robust medical transcription correction and reliable, high-stakes QA, with code released at https://github.com/RainieLLM/MedSpeak.

Abstract

Spoken question-answering (SQA) systems relying on automatic speech recognition (ASR) often struggle with accurately recognizing medical terminology. To this end, we propose MedSpeak, a novel knowledge graph-aided ASR error correction framework that refines noisy transcripts and improves downstream answer prediction by leveraging both semantic relationships and phonetic information encoded in a medical knowledge graph, together with the reasoning power of LLMs. Comprehensive experimental results on benchmarks demonstrate that MedSpeak significantly improves the accuracy of medical term recognition and overall medical SQA performance, establishing MedSpeak as a state-of-the-art solution for medical SQA. The code is available at https://github.com/RainieLLM/MedSpeak.

Figures (3)

• Figure 1: An Example of SQA.
• Figure 2: Overview of MedSpeak framework.
• Figure 3: Medical KG Constructed from SQA.