Assessing and Improving the Performance of Speech Recognition for Incremental Systems
OBS! In the seminar room
Timo Baumann - Doctoral guest researcher, Potsdam University
In incremental spoken dialogue systems, partial hypotheses about what was said are required even while the utterance is still ongoing. We define measures for evaluating the quality of incremental ASR components with respect to the _relative correctness_ of the partial
hypotheses compared to hypotheses that can optimize over the complete input, the _timing_ of hypothesis formation relative to
the portion of the input they are about, and hypothesis _stability_, defined as the number of times they are revised. We show that simple incremental post-processing can improve stability dramatically, at the cost of timeliness (from 90 % of edits of hypotheses being spurious down to 10 % at a lag of 320 ms). The measures are not independent, and we show how system designers can find a desired operating point for their ASR. To our knowledge, we are the first to suggest and examine a variety of measures for assessing incremental ASR and improve performance on this basis.