Lipreading to support Foreign and Second Language Learning by Hearing Impaired and Normally Hearing Persons
The objective of the project is to develop a syllabus and an e-learning environment to teach lipreading of foreign and second languages. Our target groups: deaf and hard of hearing adults, their therapists and teachers, and normally hearing language learners.
Over 10% of the general population in the EU has some degree of hearing loss. Hearing impaired persons depend on lipreading (also called speechreading) to help them understand a speaker. The patterns of lips and tongue help disambiguate sounds and words; facial movements and expressions support the perception and interpretation of prosodic information.
The more severe the hearing loss, the more a person will depend on lipreading to understand spoken language. Research has shown that lipreading is also used - often unconsciously - by people with perfect hearing when they try to understand a speaker under difficult circumstances: a noisy environment, and/or a language that is not the first language of either the speaker or the listener.
In foreign language teaching however, very little attention is paid to lipreading. Most deaf and severely hard-of-hearing learners learn a foreign language by reading and writing, but do not know how words are pronounced or what they look like, when spoken by a native speaker. There is a major gap between their CEFR levels for reading/writing, and for spoken interaction.
The interest in Foreign Language Lipreading training has been growing as a result of the increase in the number of people with hearing loss (a result of the aging population, and an increase of trauma caused hearing loss in young people), because of the growing number of deaf people with a Cochlear Implant, and as a result of reports that all language learners can benefit from learning tolipread.
In the LipRead project we will develop a community website, a syllabus, learning materials and a computer supported learning environment where hearing impaired people in international work environments and hearing impaired migrants can learn to lipread International English and the language of the host country.
The long term aims of the project: to raise awareness of lipreading and to improve the mobility, employability and successful entrepreneurship of hearing impaired persons.
Group: Speech Communication and Technology
Funding: EU (518970-LLP-1-2011-1-NL-KA2-KA2MP)
Duration: 2012-01-01 - 2013-12-31
Computer Speech & Language, 28(2), 607-618. [pdf] (2014). Animated Lombard speech: Motion capture, facial animation and visual intelligibility of speech produced in adverse conditions.
In Proc. of the 13th International Conference on Intelligent Virtual Agents (IVA). Edinburgh, UK. [pdf]
(2013). Web-enabled 3D talking avatars based on WebGL and HTML5. In