Grant: Strategic Projects - Group

The goal of this project is to explore mechanisms for creating alternative ways to express sound information so that people who are deaf or hard of hearing (HOH) have more equal access to sound in media. Within television and film, sound acts as a rich source of information reflecting mood, emotion, music, and sound effects, as well as dialogue. For example, people who are deaf will not necessarily realize that music is conventionally used to create tension in horror films or, if this realization is made, it may be incomplete or delayed. In using alternative means through which to communicate sound information, we want to create a parallel entertainment experience for people who are deaf or HOH. Our team seeks to translate/transfer information that is normally heard into that which is felt and/or seenThe results of this project will be an engaging and highly interactive art installation that will introduce audiences with and without disabilities to the concept of expressing emotions using alternative display techniques that combine haptic, graphics, kinetic text and video together. From a scientific perspective, we wil develop a comprehensive model that allows for the semi-automatic interpretation of emotion. The model will produce a mapping of emotion to physical parameters of the alternative displays based on the combination of the creative approach used and known techniques for detecting and categorizing emotion from specific sound sources. Not only will this have an exciting artistic outcome in new media, but also it will have important social benefits, as our model and display techniques can be used to produce more accessible audiovisual material for people who are deaf and HOH.