Non-Intrusive Automatic Speech Recognition Refinement: A Survey

TL;DR

This survey addresses non-intrusive ASR refinement, proposing a fivefold taxonomy—fusion, rescoring, correction, distillation, and training adjustment—along with domain adaptation andDataset/metrics considerations. It surveys methods from shallow/deep/cold fusion to LLM- and RAG-based corrections, distillation, and training-time objectives, highlighting practical decoding, latency, and data-access implications. The work consolidates datasets, synthetic data generation, and evaluation metrics, and identifies critical gaps: overcorrection risk, need for finer error-type analyses, richer speech-context features, and standardized benchmarks. Collectively, it provides a structured foundation to design robust, domain-aware, and efficient ASR refinement pipelines with reproducible evaluation.

Abstract

Automatic Speech Recognition (ASR) has become an integral component of modern technology, powering applications such as voice-activated assistants, transcription services, and accessibility tools. Yet ASR systems continue to struggle with the inherent variability of human speech, such as accents, dialects, and speaking styles, as well as environmental interference, including background noise. Moreover, domain-specific conversations often employ specialized terminology, which can exacerbate transcription errors. These shortcomings not only degrade raw ASR accuracy but also propagate mistakes through subsequent natural language processing pipelines. Because redesigning an ASR model is costly and time-consuming, non-intrusive refinement techniques that leave the model's architecture unchanged have become increasingly popular. In this survey, we review current non-intrusive refinement approaches and group them into five classes: fusion, re-scoring, correction, distillation, and training adjustment. For each class, we outline the main methods, advantages, drawbacks, and ideal application scenarios. Beyond method classification, this work surveys adaptation techniques aimed at refining ASR in domain-specific contexts, reviews commonly used evaluation datasets along with their construction processes, and proposes a standardized set of metrics to facilitate fair comparisons. Finally, we identify open research gaps and suggest promising directions for future work. By providing this structured overview, we aim to equip researchers and practitioners with a clear foundation for developing more robust, accurate ASR refinement pipelines.

Figures (7)

• Figure 1: A comprehensive overview of survey sections and subsections.
• Figure 2: Non-Intrusive Refinement Methods and Adaptation Techniques for Automatic Speech Recognition.
• Figure 3: Schematic of ASR refinement methods (AM and LM refer to Acoustic Model and Language Model, respectively).
• Figure 4: Schematic of Shallow Fusion in Two Consecutive Decoding Steps.
• Figure 5: Schematic of Deep Fusion in Two Consecutive Decoding Steps.