Start ReadSpeaker XT


Seminar at Speech, Music and Hearing:

Towards incremental speech production in dialogue systems

Gabriel Skantze


A traditional simplifying assumption for spoken dialogue systems is that the dialogue proceeds with strict turn-taking between user and system. The minimal unit of processing in such systems is the utterance, which is processed in whole by each module of the system before it is handed on to the next. Contrary to this, humans understand and produce language incrementally – they use multiple knowledge sources to determine when it is appropriate to speak, they give and receive backchannels in the middle of utterances, they start to speak before knowing exactly what to say, and they incrementally monitor the listener’s reactions to what they say. In this talk, I will start by presenting a new dialogue system framework for developing and experimenting with incremental dialogue systems, called Jindigo. Jindigo is partly based on the experience of developing the Higgins spoken dialogue system. However, Jindigo is implemented completely in Java and is released as an open source framework. I will then present the first steps towards a model of incremental speech production, implemented in Jindigo. The model makes it possible for the system to grab the floor before it knows exactly what to say, and to make different kinds of self-repairs, if needed. The language learning game DEAL, developed at KTH, has been re-implemented using this framework in order to do experiments on incremental speech production in a Wizard-of-Oz setting. This way, the system can start to speak when the user\'s utterance has ended, even if the Wizard hasn\'t completed the transcription of the user\'s utterance yet, resulting in a much more responsive system.

15:15 - 17:00
Friday May 28, 2010

The seminar is held in Fantum.

| Show complete seminar list

Published by: TMH, Speech, Music and Hearing

Last updated: Wednesday, 23-Jun-2010 09:22:46 MEST