Architectures for integrated multimodal input-output systems and the humanoid interface.
Kristinn Thórisson, Billund, Denmark
I will be discussing the integration of multimodal input and output in the computer interface. Such systems must be based around the idea of conversation, and borrow a lot from psychological research on turn-taking, dialogue and natural language. The focus here will be on computational mechanisms that enable the creation of such systems. While the bulk of research still deals with modes and dialouge in isolation we will try to take the full view.
Our main foundation will be the Ymir architecture, which addresses a number of difficult issues in multimodal, free-form dialogue.
Course material: Chapters 2, 3 and 4 from
Kristinn R. Thórisson Communicative Humanoids
A Computational Model of Psychosocial Dialogue Skills
Can be downloaded from:
Other interesting papers:
Thorisson (1999) A Mind Model for Communicative Creatures and Humanoids
Thorisson (1998) Decision Making in Real-Time Multimodal Dialogue
Thorisson (1997) Layered, Modular Action Control for Communicative Humanoids
Thorisson (1995) Computational Characteristics of Multimodal Dialogue
They can be downloaded from: