Grant: Operating Grant

Grasping the full meaning of a message during face-to-face conversation requires the perception of auditory and visual information. When we listen to someone speaking, we are not only processing the speech sounds but also visible speech movements, facial expression, body posture, manual gestures and the tone and timing of the voice. While we can comprehend someone's auditory speech by itself - for example, on the telephone - human communication in its most natural form is multi-sensory and multi-dimensional: when vision is available, speech intelligibility increases dramatically. The gains resulting from audiovisual integration are, however, not uniform across individuals and over our lifespan. Communication degrades with deficits in vision and audition, and these sensory deficits are common factors in communication problems associated with aging. The goal of the proposed research is to elucidate how auditory and visual speech information is combined to gain greater speech intelligibility. A series of experiments are designed to investigate how visual and auditory signals interact, how this interaction is subject to learning, and what are the underlying brain processes. This work is anticipated to provide new insights into the communication processes that giving us a social life, which could guide both the diagnosis of those eventually afflicted by communication disorders and the development of therapeutic methods to improve their social life quality, which is key to healthy aging.