Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Phonetic Error Analysis of Raw Waveform Acoustic Models with Parametric and Non-Parametric CNNs

Erfan Loweimi, Andrea Carmantini, Peter Bell, Steve Renals, Zoran Cvetkovic

TL;DR

The paper analyzes phonetic error patterns for raw waveform acoustic models on TIMIT, moving beyond standard PER by computing per-class PER across three broad phonetic categorizations and examining confusion patterns. It introduces a CNN-based architecture (parametric Sinc2Net and non-parametric variants) combined with BLSTMs, achieving state-of-the-art PER among raw waveform models and enabling transfer learning from WSJ to reduce PER further. By comparing confusion matrices with Filterbank and Wav2vec 2.0 systems, the study reveals consistent misclassification trends, notably for vowels, and highlights transfer-learning gains, especially for non-vowel classes. The findings emphasize the value of analyzing BPC-based errors to understand model limitations and guide improvements in raw waveform ASR, with implications for cross-language robustness and feature-learning strategies.

Abstract

In this paper, we analyse the error patterns of the raw waveform acoustic models in TIMIT's phone recognition task. Our analysis goes beyond the conventional phone error rate (PER) metric. We categorise the phones into three groups: {affricate, diphthong, fricative, nasal, plosive, semi-vowel, vowel, silence}, {consonant, vowel+, silence}, and {voiced, unvoiced, silence} and, compute the PER for each broad phonetic class in each category. We also construct a confusion matrix for each category using the substitution errors and compare the confusion patterns with those of the Filterbank and Wav2vec 2.0 systems. Our raw waveform acoustic models consists of parametric (Sinc2Net) or non-parametric CNNs and Bidirectional LSTMs, achieving down to 13.7%/15.2% PERs on TIMIT Dev/Test sets, outperforming reported PERs for raw waveform models in the literature. We also investigate the impact of transfer learning from WSJ on the phonetic error patterns and confusion matrices. It reduces the PER to 11.8%/13.7% on the Dev/Test sets.

Phonetic Error Analysis of Raw Waveform Acoustic Models with Parametric and Non-Parametric CNNs

TL;DR

The paper analyzes phonetic error patterns for raw waveform acoustic models on TIMIT, moving beyond standard PER by computing per-class PER across three broad phonetic categorizations and examining confusion patterns. It introduces a CNN-based architecture (parametric Sinc2Net and non-parametric variants) combined with BLSTMs, achieving state-of-the-art PER among raw waveform models and enabling transfer learning from WSJ to reduce PER further. By comparing confusion matrices with Filterbank and Wav2vec 2.0 systems, the study reveals consistent misclassification trends, notably for vowels, and highlights transfer-learning gains, especially for non-vowel classes. The findings emphasize the value of analyzing BPC-based errors to understand model limitations and guide improvements in raw waveform ASR, with implications for cross-language robustness and feature-learning strategies.

Abstract

In this paper, we analyse the error patterns of the raw waveform acoustic models in TIMIT's phone recognition task. Our analysis goes beyond the conventional phone error rate (PER) metric. We categorise the phones into three groups: {affricate, diphthong, fricative, nasal, plosive, semi-vowel, vowel, silence}, {consonant, vowel+, silence}, and {voiced, unvoiced, silence} and, compute the PER for each broad phonetic class in each category. We also construct a confusion matrix for each category using the substitution errors and compare the confusion patterns with those of the Filterbank and Wav2vec 2.0 systems. Our raw waveform acoustic models consists of parametric (Sinc2Net) or non-parametric CNNs and Bidirectional LSTMs, achieving down to 13.7%/15.2% PERs on TIMIT Dev/Test sets, outperforming reported PERs for raw waveform models in the literature. We also investigate the impact of transfer learning from WSJ on the phonetic error patterns and confusion matrices. It reduces the PER to 11.8%/13.7% on the Dev/Test sets.
Paper Structure (8 sections, 2 equations, 6 figures, 3 tables)

This paper contains 8 sections, 2 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Raw waveform Acoustic Modelling
  4. Applications of BPCs
  5. Architecture
  6. Phonetic Categorisations
  7. Experimental Results and Discussion
  8. Conclusion

Figures (6)

  • Figure 1: Our raw waveform acoustic models consist of a cascade of (parametric or non-parametric) convolutional, BLSTM and fully-connected (FC) layers. The output layer composed of context-dependent (CD) and context-independent (CI) heads.
  • Figure 2: Recognition errors for FBank and raw waveform models. (a) PER, (b) Sub, (C) Del, (e) Ins.
  • Figure 3: PER for FBank and raw waveform models. (a) Consonant/Vowel$^+$/Silence, (b) Voiced/Unvoiced/Silence.
  • Figure 4: Confusion matrices of three phonetic categorisations for Sinc2Net on TIMIT's Dev set. The bold and underlined numbers denote the first and second mostly confused classes.
  • Figure 5: Confusion matrices of three phonetic categorisations for RawWav-CNN-WSJ on TIMIT's Dev set. The bold and underlined denote the first and second mostly confused classes.
  • ...and 1 more figures