Fewer Hallucinations, More Verification: A Three-Stage LLM-Based Framework for ASR Error Correction

TL;DR

<3-5 sentence high-level summary> The paper addresses the challenge of correcting ASR errors without inducing hallucinations by leveraging general-purpose LLMs. It introduces the Reliable LLM Correction Framework (RLLM-CF), a three-stage approach comprising error pre-detection, chain-of-thought subtask iterative correction, and answer verification, designed to operate without fine-tuning or external data. Empirical results on AISHELL-1, AISHELL-2, and LibriSpeech show notable reductions in $CER$ and $WER$ across languages, with explicit analysis of hallucination types and ablations that validate the framework’s contributions. The work offers a practical, scalable path for robust ASR error correction using LLMs in real-world settings, emphasizing reliability and grounded reasoning over naive correction.

Abstract

Automatic Speech Recognition (ASR) error correction aims to correct recognition errors while preserving accurate text. Although traditional approaches demonstrate moderate effectiveness, LLMs offer a paradigm that eliminates the need for training and labeled data. However, directly using LLMs will encounter hallucinations problem, which may lead to the modification of the correct text. To address this problem, we propose the Reliable LLM Correction Framework (RLLM-CF), which consists of three stages: (1) error pre-detection, (2) chain-of-thought sub-tasks iterative correction, and (3) reasoning process verification. The advantage of our method is that it does not require additional information or fine-tuning of the model, and ensures the correctness of the LLM correction under multi-pass programming. Experiments on AISHELL-1, AISHELL-2, and Librispeech show that the GPT-4o model enhanced by our framework achieves 21%, 11%, 9%, and 11.4% relative reductions in CER/WER.

Figures (4)

• Figure 1: Overview of the Reliable LLM Correction Framework (RLLM-CF). The blue components represent modules executed by the LLM, while the yellow components are handled by the external program. The green block indicates the recognition result, and the black block denotes the final output. The entire error correction process follows the direction of the arrows.
• Figure 2: The above picture introduces the specific parts of Prompt's design in detail.;The text in the white boxes represents the specific prompt content, while the blue boxes provide explanations.
• Figure 3: The above is a detailed answer to illustrate the composition and specific answers of the subtasks.
• Figure 4: Analysis of Results by Sentence on AISHELL-1 Using DeepSeek-V2 and Attention Decoding Method; The ASR model is a conformer-based AED model trained on AISHELL-1