FlanEC: Exploring Flan-T5 for Post-ASR Error Correction

TL;DR

This work investigates FlanEC, an instruction-tuned Flan-T5 encoder-decoder for post-ASR error correction (GenSEC), by mapping $n$-best ASR hypotheses to a single corrected transcription. It systematically assesses model scale (250M–3B), data strategy (SD vs CD), and adaptation methods (LoRA vs full fine-tuning) on the HyPoradise benchmark, highlighting cross-domain generalization and efficiency trade-offs. Across eight ASR domains, cumulative training with full fine-tuning on larger models yields the strongest improvements in WER, though CORAAL remains particularly challenging due to token-level divergences not captured by $n$-best lists. The results demonstrate the value of instruction-tuned encoder-decoder models for GenSEC and point to scalable, domain-diverse training as a path toward robust post-ASR correction in real-world applications.

Abstract

In this paper, we present an encoder-decoder model leveraging Flan-T5 for post-Automatic Speech Recognition (ASR) Generative Speech Error Correction (GenSEC), and we refer to it as FlanEC. We explore its application within the GenSEC framework to enhance ASR outputs by mapping n-best hypotheses into a single output sentence. By utilizing n-best lists from ASR models, we aim to improve the linguistic correctness, accuracy, and grammaticality of final ASR transcriptions. Specifically, we investigate whether scaling the training data and incorporating diverse datasets can lead to significant improvements in post-ASR error correction. We evaluate FlanEC using the HyPoradise dataset, providing a comprehensive analysis of the model's effectiveness in this domain. Furthermore, we assess the proposed approach under different settings to evaluate model scalability and efficiency, offering valuable insights into the potential of instruction-tuned encoder-decoder models for this task.