Start ReadSpeaker XT


Seminar at Speech, Music and Hearing:

Affordance based word-to-meaning association

Giampiero Salvi


In this seminar I will present one of the studies carried out during my stay at the Vislab, Instituto Superior Técnico, Lisbon, Portugal. The study introduces a method to associate meanings to words in robotic manipulation tasks. The model is based on an affordance network, i.e., a Bayesian network used to map between robot actions, robot perceptions and the perceived effects of these actions upon objects. We extend the affordance model to incorporate words. Using verbal descriptions of a task, the model uses co-occurrence to create links between speech utterances and the involved objects, actions and effects. We show that the robot is able to form useful word-to-meaning associations, even without considering grammatical structure in the learning process and in the presence of recognition errors. These word-to-meaning associations are embedded in the robot´s own understanding of its actions. Thus they can be directly used to instruct the robot to perform tasks and also allow to incorporate context in the speech recognition task.

Key-words: speech, manipulation, unsupervised learning, Bayesian network, affordances

15:15 - 17:00
Friday November 28, 2008

The seminar is held in Fantum.

| Show complete seminar list

Published by: TMH, Speech, Music and Hearing

Last updated: Wednesday, 23-Jun-2010 09:22:46 MEST