THE TELEFACE PROJECT

Get updated!

---

Overview

The Teleface project at the Department of Speech, Music and Hearing, KTH, aims at evaluating the possibilities of using synthetic visual speech in tools for hearing-impaired people. The project will include an effort to implement a demonstrator prototype of a telephone communication aid for the hard of hearing. This device will generate a synthetic face that articulates in synchrony with the telephone speech using only the information contained in the telephone speech signal, see Figure 1 and video sequence. A longer version MPEG (22.7 Mb) presenting the background and the goal for the Teleface project is also available for download.

Here's a new video clip that shows one of our talking heads with speech movements generated from the audio using a HMM recognizer. As you can see it is speaker independant, and to a certain extent it works with other languages also (Example in english)

The project involves work in several areas: multimodal speech synthesis, audiovisual speech intelligibility studies, visual speech measurement and speech signal analysis. We are currently performing a suite of audio-visual intelligibility tests, where cross-combinations of synthetic and natural voices and faces are evaluated at varying signal-to-noise ratios. Subjects are hearing impaired as well as normal hearing people.

The Teleface project is funded by KFB, the Swedish Transport and Communications Research Board.
 

Figure 1: The intended teleface demonstrator. Illustration by Viera Larsson

---

Selected publications and presentations

Öhman, T. (2000) "Vision in Speech Technology. Automatic measurements of visual speech and audio-visual intelligibility of synthetic and natural faces." Licentiate Thesis, TMH, KTH, 2000.

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "Synthetic visual speech driven from auditory speech" In Proceedings from AVSP'99, Santa Cruz, USA.

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "Two methods for Visual Parameter Extraction in the Teleface Project" in Proceedings from Fonetik 99, Gothenburg, Sweden.

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "Artificial video for Hearing-Impaired Telephone Users; A comparison with the No Video and Perfect Video Conditions." in Assitive Technology on the Threshold of the New Millenium, Assistive Technology Research Series 6. Eds Christian Buhler und Harry Knops, ISBN 1 586030019 (IOS Press)

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "A Synthetic Face as a Lip-reading Support for Hearing Impaired Telephone Users - Problems and Positive Results". EFAS 1999, 4th European Conference on Audiology, June 6-10th 1999, Oulo, Finland

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1999). "Teleface - syntetiskt ansikte som avläsestöd för hörselskadade telefonanvändare" Sveriges Tekniska Audiologers Förening, STAF 99 in Lund, Sweden.

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1999). "ASR controlled synthetic face as a lipreading support for hearing impaired telephone users" Cost249 meeting Feb'99 in Prague, Czech Republic.

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1998). "Teleface - the use of a synthetic face for the hard of hearing". To appear in: Proceedings of IVTTA'98, Torino, Italia. []

Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1998). "The synthetic face in a hearing impaired view". In Proceedings of Fonetik'98, Stockholm, Sweden. [HTML]

Beskow, J.,Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1997). "The Teleface project - Multimodal Speech Communication for the Hearing Impaired". in Proceedings of Eurospeech '97, Rhodos, Greece. [postscript][gzipped postscript] More results presented at Eurospeech'97 [html]

Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E. & Öhman, T. (1997): "The Teleface project - disability, feasibility and intelligibility" in Proceedings of Fonetik -97, Swedish Phonetics Conference, Umeå, Sweden [html][postscript][gzipped postscript]

List of publications from the multimodal speech technology group.

---

People

• Karl-Erik Spens
• Björn Granström
• Magnus Lundeberg
• Jonas Beskow
• Martin Dahlquist
• Eva Agelfors
• Giampero Salvi
• Tobias Öhman