HypR: A comprehensive study for ASR hypothesis revising with a reference corpus

TL;DR

To address inconsistent evaluation in ASR hypothesis revising, the paper introduces HypR, a unified benchmark aggregating AISHELL-1, TED-LIUM 2, and LibriSpeech with 50 hypotheses per utterance and released ASR checkpoints. It surveys two main hypothesis revising paradigms—N-best reranking and error correction—along with representative methods across token-level, sentence-level, and comparison-based categories, and also explores prompt-based large-language-model usage. Empirical results show that sentence-level models like PBERT+LSTM perform strongly among rerankers, while error-correction gains depend on hypothesis quality, and current LLM-based strategies offer limited, context-dependent improvements. Overall, HypR provides a reproducible platform for fair comparisons and guides future work toward integrated acoustic-lexical strategies and broader domain coverage.

Abstract

With the development of deep learning, automatic speech recognition (ASR) has made significant progress. To further enhance the performance of ASR, revising recognition results is one of the lightweight but efficient manners. Various methods can be roughly classified into N-best reranking modeling and error correction modeling. The former aims to select the hypothesis with the lowest error rate from a set of candidates generated by ASR for a given input speech. The latter focuses on detecting recognition errors in a given hypothesis and correcting these errors to obtain an enhanced result. However, we observe that these studies are hardly comparable to each other, as they are usually evaluated on different corpora, paired with different ASR models, and even use different datasets to train the models. Accordingly, we first concentrate on providing an ASR hypothesis revising (HypR) dataset in this study. HypR contains several commonly used corpora (AISHELL-1, TED-LIUM 2, and LibriSpeech) and provides 50 recognition hypotheses for each speech utterance. The checkpoint models of ASR are also published. In addition, we implement and compare several classic and representative methods, showing the recent research progress in revising speech recognition results. We hope that the publicly available HypR dataset can become a reference benchmark for subsequent research and promote this field of research to an advanced level.