Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RECOVER: Robust Entity Correction via agentic Orchestration of hypothesis Variants for Evidence-based Recovery

Abhishek Kumar, Aashraya Sachdeva

Abstract

Entity recognition in Automatic Speech Recognition (ASR) is challenging for rare and domain-specific terms. In domains such as finance, medicine, and air traffic control, these errors are costly. If the entities are entirely absent from the ASR output, post-ASR correction becomes difficult. To address this, we introduce RECOVER, an agentic correction framework that serves as a tool-using agent. It leverages multiple hypotheses as evidence from ASR, retrieves relevant entities, and applies Large Language Model (LLM) correction under constraints. The hypotheses are used using different strategies, namely, 1-Best, Entity-Aware Select, Recognizer Output Voting Error Reduction (ROVER) Ensemble, and LLM-Select. Evaluated across five diverse datasets, it achieves 8-46% relative reductions in entity-phrase word error rate (E-WER) and increases recall by up to 22 percentage points. The LLM-Select achieves the best overall performance in entity correction while maintaining overall WER.

RECOVER: Robust Entity Correction via agentic Orchestration of hypothesis Variants for Evidence-based Recovery

Abstract

Entity recognition in Automatic Speech Recognition (ASR) is challenging for rare and domain-specific terms. In domains such as finance, medicine, and air traffic control, these errors are costly. If the entities are entirely absent from the ASR output, post-ASR correction becomes difficult. To address this, we introduce RECOVER, an agentic correction framework that serves as a tool-using agent. It leverages multiple hypotheses as evidence from ASR, retrieves relevant entities, and applies Large Language Model (LLM) correction under constraints. The hypotheses are used using different strategies, namely, 1-Best, Entity-Aware Select, Recognizer Output Voting Error Reduction (ROVER) Ensemble, and LLM-Select. Evaluated across five diverse datasets, it achieves 8-46% relative reductions in entity-phrase word error rate (E-WER) and increases recall by up to 22 percentage points. The LLM-Select achieves the best overall performance in entity correction while maintaining overall WER.
Paper Structure (11 sections, 1 equation, 1 figure, 3 tables)

This paper contains 11 sections, 1 equation, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Approach
  3. Multi-Hypothesis Generation
  4. Dynamic Entity Candidate Retrieval
  5. Agentic Orchestration
  6. Experimental Setup
  7. Datasets
  8. ASR system and LLM
  9. Evaluation
  10. Results and Discussion
  11. Conclusions

Figures (1)

  • Figure 1: End-to-end system overview. Audio is decoded at five temperatures using Whisper-small, producing five transcript hypotheses. Entity phrases are scored against the hypotheses using exact, fuzzy, and phonetic matching and the top-$K$ candidates ($K\!=\!200$) are selected. The agentic orchestrates three tools in sequence: Tool 1 (Fuse Hypotheses), Tool 2 (Propose Corrections via a constrained LLM prompt), and Tool 3 (Verify & Apply with deterministic guardrails), producing the corrected transcript.