Grant: Discovery Grants Program - Individual
University of British Columbia
$40,000.00
Concurrency, synchrony, and interaction in multimodal communication
EF 2007-08
Résumé
Human-machine interaction is becoming more and more prevalent in our lives. Eventually, we predict that such interaction will be linguistically communicative and will entail machine emulation of a human conversant. Simple interfaces such as voice recognition over the telephone will be replaced by interaction with physical agents operating in multiple signal/sensory modalities. An obvious question is how well will humans and machines be able to communicate if their respective domains of behavior are matched to a fair degree. In order to do this, we need to know what the bases of multisensory communication are for humans, so that we can insure that machines can operate within the necessary tolerances of human processing. This research uses artificial devices such as a robot jaw, animated talking head, and a talking glove, along with distortions of human behavior to explore the range of distortions the communicative process can withstand at the levels of the production and processing of signals in auditory and visual modalities. Structural (face shape), functional (replacing the face with a hand), and behavioral (loudness) exaggerations in spoken behavior are used to characterize how the audible and visible components correspond and co-depend on one another. We call this correspondence and co-dependency between signals and structures "coherence". The goal is to determine how much distortion the human processing system can withstand before communicative interaction breaks down. Examining the effects of these distortions on both the production and perception of spoken communication will reveal what works in human-machine interaction and bring us closer to an understanding of how it works.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
261549-2007
Name nom
VatikiotisBateson, Eric
Department département
Linguistics
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2007
Competition year année de concours
2007
Award amount
40000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
330
Committee name en
Computing and Information Sciences - A
Committee name fr
Informatique - A
Area of application code
405
Area of application group en
Environment
Area of application group fr
Environnement
Area of application en
Modelling and mathematical simulation of natural processes
Area of application fr
Modélisation et simulation mathématique des processus naturels
Research subject code
2606
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Systems, man and cybernetics
Research subject fr
Systèmes, homme et cybernétique
Partie
1
Nb partie
5
Application title
Concurrency, synchrony, and interaction in multimodal communication
Keyword
NA
Application summary
Human-machine interaction is becoming more and more prevalent in our lives. Eventually, we predict that such interaction will be linguistically communicative and will entail machine emulation of a human conversant. Simple interfaces such as voice recognition over the telephone will be replaced by interaction with physical agents operating in multiple signal/sensory modalities. An obvious question is how well will humans and machines be able to communicate if their respective domains of behavior are matched to a fair degree. In order to do this, we need to know what the bases of multisensory communication are for humans, so that we can insure that machines can operate within the necessary tolerances of human processing. This research uses artificial devices such as a robot jaw, animated talking head, and a talking glove, along with distortions of human behavior to explore the range of distortions the communicative process can withstand at the levels of the production and processing of signals in auditory and visual modalities. Structural (face shape), functional (replacing the face with a hand), and behavioral (loudness) exaggerations in spoken behavior are used to characterize how the audible and visible components correspond and co-depend on one another. We call this correspondence and co-dependency between signals and structures "coherence". The goal is to determine how much distortion the human processing system can withstand before communicative interaction breaks down. Examining the effects of these distortions on both the production and perception of spoken communication will reveal what works in human-machine interaction and bring us closer to an understanding of how it works.