THE TELEFACE PROJECT
Multimodal speech communication
- Telephone Application with Synthetic Speechreading Support for Hearing
Impaired
Get updated!
The Teleface project ended in 2002 according to plans. A new project, funded by the European Commission was then started with the name Synface. Go to the Synface official home-page for more information.
Overview
The Teleface project at the Department
of Speech, Music and Hearing, KTH,
aims at evaluating the possibilities of using synthetic visual speech in
tools for hearing-impaired people. The project will include an effort to
implement a demonstrator prototype of a telephone communication aid for
the hard of hearing. This device will generate a synthetic face
that articulates in synchrony with the telephone speech using only the
information contained in the telephone speech signal, see Figure
1 and video sequence. A longer version MPEG (22.7 Mb) presenting the background and the goal for the Teleface project is also available for download.
Here's a new video clip that shows one of our talking heads with speech movements generated from the audio using a HMM recognizer. As you can see it is speaker independant, and to a certain extent it works with other languages also (Example in english)
The project involves work in several areas: multimodal
speech synthesis, audiovisual speech intelligibility studies, visual
speech measurement and speech signal analysis. We are currently performing
a suite of audio-visual intelligibility tests, where cross-combinations
of synthetic and natural voices and faces are evaluated at varying signal-to-noise
ratios. Subjects are hearing impaired as well as normal hearing people.
The Teleface project is funded by KFB,
the Swedish Transport and Communications Research Board.
 |
| Figure 1: The intended teleface demonstrator. Illustration by Viera Larsson |
Selected publications and presentations
Öhman, T. (2000) "Vision in Speech Technology. Automatic measurements of visual speech and audio-visual intelligibility of synthetic and natural faces." Licentiate Thesis, TMH, KTH, 2000.
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "Synthetic visual speech driven from auditory speech" In Proceedings from AVSP'99, Santa Cruz, USA.
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "Two methods for Visual Parameter Extraction in the Teleface Project" in Proceedings from Fonetik 99, Gothenburg, Sweden.
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "Artificial video for Hearing-Impaired Telephone Users; A comparison with the No
Video and Perfect Video Conditions." in Assitive Technology on the Threshold of the New
Millenium, Assistive Technology Research Series 6. Eds Christian Buhler und Harry Knops, ISBN 1
586030019 (IOS Press)
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Salvi, G., Spens, K-E & Öhman, T. (1999). "A Synthetic Face as a Lip-reading Support for Hearing Impaired Telephone Users -
Problems and Positive Results". EFAS 1999, 4th European Conference on Audiology, June 6-10th 1999,
Oulo, Finland
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1999). "Teleface - syntetiskt ansikte som avläsestöd för hörselskadade telefonanvändare" Sveriges Tekniska Audiologers Förening, STAF 99 in Lund, Sweden.
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1999). "ASR controlled synthetic face as a lipreading support for hearing impaired telephone users" Cost249 meeting Feb'99 in Prague, Czech Republic.
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg,
M., Spens, K-E & Öhman, T. (1998). "Synthetic faces as a lipreading
support". To appear in: Proceedings of ICSLP'98, Sydney, Australia. [HTML]
[postscript]
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg,
M., Spens, K-E & Öhman, T. (1998). "Teleface - the use of
a synthetic face for the hard of hearing". To appear in: Proceedings
of IVTTA'98, Torino, Italia. []
Agelfors, E., Beskow, J., Dahlquist, M., Granström, B., Lundeberg,
M., Spens, K-E & Öhman, T. (1998). "The synthetic face in
a hearing impaired view". In Proceedings of Fonetik'98, Stockholm,
Sweden. [HTML]
Beskow, J.,Dahlquist, M., Granström, B., Lundeberg, M., Spens,
K-E & Öhman, T. (1997). "The Teleface project - Multimodal
Speech Communication for the Hearing Impaired". in Proceedings of Eurospeech
'97, Rhodos, Greece. [postscript][gzipped
postscript] More results
presented at Eurospeech'97 [html]
Beskow, J., Dahlquist, M., Granström, B., Lundeberg, M., Spens,
K-E. & Öhman, T. (1997): "The Teleface project - disability,
feasibility and intelligibility" in Proceedings of Fonetik -97, Swedish
Phonetics Conference, Umeå, Sweden [html][postscript][gzipped
postscript]
List of publications
from the multimodal speech technology group.
People
The following people have been involved in the Teleface project:
|
