Post-decoder Biasing for End-to-End Speech Recognition of Multi-turn Medical Interview

TL;DR

This work presents Medical Interview (MED-IT), a multi-turn consultation speech dataset that contains a substantial number of knowledge-intensive named entities, and proposes a novel approach, post-decoder biasing, which constructs a transform probability matrix based on the distribution of training transcriptions.

Abstract

End-to-end (E2E) approach is gradually replacing hybrid models for automatic speech recognition (ASR) tasks. However, the optimization of E2E models lacks an intuitive method for handling decoding shifts, especially in scenarios with a large number of domain-specific rare words that hold specific important meanings. Furthermore, the absence of knowledge-intensive speech datasets in academia has been a significant limiting factor, and the commonly used speech corpora exhibit significant disparities with realistic conversation. To address these challenges, we present Medical Interview (MED-IT), a multi-turn consultation speech dataset that contains a substantial number of knowledge-intensive named entities. We also explore methods to enhance the recognition performance of rare words for E2E models. We propose a novel approach, post-decoder biasing, which constructs a transform probability matrix based on the distribution of training transcriptions. This guides the model to prioritize recognizing words in the biasing list. In our experiments, for subsets of rare words appearing in the training speech between 10 and 20 times, and between 1 and 5 times, the proposed method achieves a relative improvement of 9.3% and 5.1%, respectively.

Figures (4)

• Figure 1: Creation pipeline of MED-IT. Data collection was done in published research. Cleaning and segmentation on both modalities have been performed serialized for textual-speech alignment, and then manual examination for exception handling.
• Figure 2: Statistics of our dataset. (a) shows the speech duration of each department with RES making the most. (b) shows the number of seconds per utterance, with most lasting for less than 10 seconds. (c) indicates the partition portion of each biasing list, and (d) shows word slices from them.
• Figure 3: Overview of the post-decoder biasing for attention-based encoder-decoder. The transcription of the train set contains the biasing word "warfarin". BPE "rant" might be transformed to "farin" if the corresponding probability is abnormally high, for "warrant" and "warfarin" are both valid words. The biasing decoding results are determined by both neural architecture and transform matrix obtained from the in-domain sub-word combination distribution.
• Figure 4: Relative rare words recognition improvement with different enhancing frequencies. Negative values indicate a certain degree of decline compared to the baseline.