AdApt's Logotype

A multimodal spoken dialogue system for browsing apartments on the Stockholm real estate market

There is an incomplete bibliography at the end of this page

Introduction

The AdApt project, which ran in the years 1998 to 2002, had as its goal to be the foundation for the development and evaluation of advanced multimodala spoken dialogue systems. Within the project, a spoken dialogue system where a user could cooperate with an animated talking agent to solve more complex problems than what had previously been achieved in our systems.
The project was a joint project between TMH and Telia Research within the framework of CTT (the Centre for Speech Technology), and engaged, amongst others, the following researchers:
TMH/CTT
Linda Bell (currently TeliaSonera), Jonas Beskow, Rolf Carlson, Jens Edlund, Joakim Gustafsson (currently TeliaSonera), Anna Hjalmarsson, Magnus Nordstrand
Telia Research
Johan Boye, Mats Wirén
The domain chosen was one where multimodal communication matters, and one that engaged a broad audience: the real estate market in downtown Stockholm.
In the Adapt system, the compuiterperson plays a part close to that of the real estate broker: to help people find apartments, to describe apartments, to reply to and answer questions, and to lend support by finding information in apartment ads.
The information used by the system comes from authentic apartment ads published on the internet.
In addition to spoken input, the user can provide information by clicking or marking areas on an interactive map over downtown Stockholm. The system output consists of a talking, animated head and graphical animated icons on the interactive map. The system is also capable of presenting information as text in the form of tables, although this is rarely used. There is a lot more info on the animated talking head on our multimodal speech synthesis pages.
The AdApt user interface
This is a screen dump from an early version of the system interface. The image contains Urban, the interactive map, icons, and a table.
There are a couple of films featuring the animated agent Urban below:
The Adapt project was features in the exhibition Fritt Fram which ran from 2000 to 2001 in Tekniska Museet in Stockholm, together with several of the TMH/CTT talking heads.
Talking Hedas at the Fritt Fram exhibition
The picture shows the Talking Heads we displayed at Fritt Fram.

Background

AdApt was a natural continuation of previous dialogue projects at TMH and CTT, in that it placed higher and different demands on the technologies. It used live sophisticated information extraction (interpretation of open decsriptions of apartments in internet apartment ads), storage and structuring of information (all highly generalised; seamless conversion between formats, e.g. Prolog facts and XML structures), semantic representation of information and utterances, and generation of utterances where facial expressions, stress, etc. is tantamount for the interpretation. The system also features a dialogue manager that can handle a number of misunderstandings in a sensible manner. Notable predecessors include the August and Waxholm systems. For an overview, see Joakim Gustafson's doctoral thesis.

Future

Many of the participants in the AdApt project are active in new spoken dialogue system projects. At TeliaSonera (former Telia Research), the AdApt researchers have completed the Pixie and Nice projects, and at TMH/CTT, the Higgins project is an immediate successor of AdApt and our research in the CHIL project draws heavily on experiences from AdApt.

Comments?

Contact Jens (edlund@speech.kth.se).

Bibliography