Segment Boundary Detection via Class Entropy Measurements in Connectionist Phoneme Recognition

TL;DR

The paper addresses predicting phoneme boundary times by exploiting the entropy of posterior class probabilities produced by a connectionist phoneme recogniser. It evaluates multiple measures based on entropy and its time derivatives, including a neural-network fusion approach, against simple thresholds and baseline MAP changes, using 10 and 20 ms evaluation windows. The nonlinear combination of entropy with its derivatives via a time-delayed neural network yields the strongest performance, achieving up to 75.0% precision and 64.5% recall at 10 ms (86.4% and 76.2% at 20 ms), with forced alignment providing an upper bound on achievable accuracy. The work demonstrates practical viability for boundary detection in speech applications, such as lip-synchronization in avatars, while highlighting limitations due to frame-level resolution and evaluation criteria.

Abstract

This article investigates the possibility to use the class entropy of the output of a connectionist phoneme recogniser to predict time boundaries between phonetic classes. The rationale is that the value of the entropy should increase in proximity of a transition between two segments that are well modelled (known) by the recognition network since it is a measure of uncertainty. The advantage of this measure is its simplicity as the posterior probabilities of each class are available in connectionist phoneme recognition. The entropy and a number of measures based on differentiation of the entropy are used in isolation and in combination. The decision methods for predicting the boundaries range from simple thresholds to neural network based procedure. The different methods are compared with respect to their precision, measured in terms of the ratio between the number C of predicted boundaries within 10 or 20 msec of the reference and the total number of predicted boundaries, and recall, measured as the ratio between C and the total number of reference boundaries.

Figures (7)

• Figure 1: Box plot of the frame entropy for each phonemic class. The maximum entropy is $\log_2 50 = 5.64$ bits. The SAMPA phonetic symbols gs:Wells1997 are used with a few exceptions gs:Lindberg2000: } $\rightarrow$ uh, 2 $\rightarrow$ ox, { $\rightarrow$ ae, 9 $\rightarrow$ oe, @ $\rightarrow$ eh
• Figure 2: Illustration of the time span of each measure with respect to the candidate boundary
• Figure 3: Illustration of the $\stackrel{\star}{>}$ decision method: given a measure $m$, one boundary candidate is selected for each region of contiguous frames for which $m>\hbox{th}$. The candidate is the global maximum for $m$ in that region. The vertical arrows indicate the resulting boundaries, the cross corresponds to a possible deletion.
• Figure 4: Example of test utterance containing the phrase "jag vet vad svenskarna kan" ([jA:ve:tvA:svenskarnakan]. The last four plots show different measures described in the text (continuous line), the reference boundaries (dotted lines) and the predicted boundaries (arrows). The last plot also include the proximity function (dotted line) used as target for the neural network training.
• Figure 5: Threshold optimisation for method $\mathbf{e[n]+e^{"}[n]}$ (a,b) and method $\mathbf{e[n]+\hbox{\bf ma}[n]}$ (c,d). Plots (a) and (c) show the recall $R$ (surface with the more pronounced maximum) and the precision $P$ as a function of the two thresholds th1 and th2. The 100% plane is also shown in the plots. Plots (b) and (d) show the optimality criterion crit as a function of th1 and th2.