Architectures for integrated multimodal input-output systems and the humanoid interface.

Kristinn Thórisson, Billund, Denmark


I will be discussing the integration of multimodal input and output in the computer interface. Such systems must be based around the idea of conversation, and borrow a lot from psychological research on turn-taking, dialogue and natural language. The focus here will be on computational mechanisms that enable the creation of such systems. While the bulk of research still deals with modes and dialouge in isolation we will try to take the full view.

Our main foundation will be the Ymir architecture, which addresses a number of difficult issues in multimodal, free-form dialogue.


Course material: Chapters 2, 3 and 4 from

Kristinn R. Thórisson Communicative Humanoids
A Computational Model of Psychosocial Dialogue Skills

Can be downloaded from:



Other interesting papers:

Thorisson (1999) A Mind Model for Communicative Creatures and Humanoids

Thorisson (1998) Decision Making in Real-Time Multimodal Dialogue

Thorisson (1997) Layered, Modular Action Control for Communicative Humanoids

Thorisson (1995) Computational Characteristics of Multimodal Dialogue


They can be downloaded from: