Situated AV-Interaction with Robots

Strategic Research Area ICT - The Next Generation

Documents

Situated Audio Visual Interaction with Robots is a project within the Strategic Research Area ICT - The Next Generation.

The goal of the project is to build a research platform which combines spoken dialogue technology with visual object recognition in a robot. It will enable research on:

true mixed initiative, collaborative human-robot interaction.
cognitive modeling of visual scenes
learning by audiovisual interaction with humans

The robot should with the help of a human be able to:

acquire new knowledge
learn new skills
adapt the actions to the assisted user
adapt to the possibly changing environment

The project core directions, audiovisual interaction and robotics, is a challenging developing research area.

KTH is unique in having successful groups complementing each other through their backgrounds in each respective research discipline. The groups are engaged in robotics-related EU and VR projects such as IURO, GRASP, PACO-PLUS and CogX.

Groups and Researchers

Department of Speech, Music and Hearing (TMH), KTH

Computer Vision and Active Perception Lab (CVAP), KTH

Automatic Control Lab (ACCESS), KTH

Interaction Design and Innovation (IDI), SICS

Lars Erik Holmquist

Video demonstration

Publications

Johnson-Roberson, M., Bohg, J., Skantze, G., Gustafson, J., Carlson, R., Rasolzadeh, B., & Kragic, D. (2011). Enhanced Visual Scene Understanding through Human-Robot Dialog. In IEEE/RSJ International Conference on Intelligent Robots and Systems. [pdf]Johnson-Roberson, M., Bohg, J., Kragic, D., Skantze, G., Gustafson, J., & Carlson, R. (2010). Enhanced Visual Scene Understanding through Human-Robot Dialog. In Proceedings of AAAI 2010 Fall Symposium: Dialog with Robots. Arlington, VA. [pdf]