Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Simon Fraser University

$39,000.00

Adaptations in conversation: engaging voices, faces, brains and machines

EF 2024-25

Résumé
Speaking can involve multiple ways of communicating. We are routinely engaged in face-to-face conversations, voice-only phone chats, and increasingly, video-based online communication. We interact with people with different language backgrounds, and nowadays even with AI chatbots. Our experiences thus involve adjustments in our speech production and perception based on whom we communicate with and in what environments. For example, we may speak more clearly than usual when talking with non-native speakers or in a noisy cafeteria, or we may accommodate to our conversation partner's speech style or variation. One question thus arises is how effective our adaptations are. In this research, we explore how conversation partners adjust speech for successful communication. Specifically, we ask how we work together to accommodate each other during spontaneous interactions. What kinds of adaptations are helpful? Do effects of adaptation differ for face-to-face versus voice-only conversations? Do we adapt differently communicating with AI vs. human partners? Does brain activity differ as a function of how we accommodate speech in conversations? What are the relative contributions of different articulatory, acoustic, perceptual, and neural aspects of adaptation in effective communication? To address these questions, we combine behavioral, neural and computational approaches to examine audio-video recordings of live conversations involving interactive tasks to elicit words with specific sound contrasts. Videos of speakers' facial movements during target sound productions and acoustic correlates of the same productions will be analyzed, along with how these differences are perceived. Neural data will also be collected through functional neuroimaging to study the brain activity for both conversation partners simultaneously. Finally, articulatory, acoustic, perceptual, and neural data will be brought together using computational modeling to develop prediction for which attributes improve the likelihood of accurate communication. This research has significant implications in speech science, communication, neuroscience, computer speech technology and speech engineering. Findings will inform how speech adaptation principles can be applied in the development of technologies for adverse listening or visual environments, speech and hearing therapy, language learning and teaching, and human-computer interfaces. We expect this research to contribute to new knowledge leading to clearer channels of communication in Canada's multi-lingual and multimedia society.
Programme
Discovery Grants Program - Individual
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2023-05666
Name nom
Wang, Yue
Department département
Linguistics
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
39000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1502
Committee name en
Biological Systems and Functions
Committee name fr
Systèmes et fonctions biologiques
Area of application code
1210
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Psychology
Area of application fr
Psychologie
Research subject code
6306
Research subject group en
Psychology
Research subject group fr
Psychologie
Research subject en
Cognitive science -- language
Research subject fr
Science cognitive -- langage
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Adaptations in conversation: engaging voices, faces, brains and machines
Keywords
phonetic adaptation; conversation; auditory visual speech; speech style; speech intelligibility; human-machine communication; interactive brain; computational modeling; multi-modal processing
Application summary
Speaking can involve multiple ways of communicating. We are routinely engaged in face-to-face conversations, voice-only phone chats, and increasingly, video-based online communication. We interact with people with different language backgrounds, and nowadays even with AI chatbots. Our experiences thus involve adjustments in our speech production and perception based on whom we communicate with and in what environments. For example, we may speak more clearly than usual when talking with non-native speakers or in a noisy cafeteria, or we may accommodate to our conversation partner's speech style or variation. One question thus arises is how effective our adaptations are. In this research, we explore how conversation partners adjust speech for successful communication. Specifically, we ask how we work together to accommodate each other during spontaneous interactions. What kinds of adaptations are helpful? Do effects of adaptation differ for face-to-face versus voice-only conversations? Do we adapt differently communicating with AI vs. human partners? Does brain activity differ as a function of how we accommodate speech in conversations? What are the relative contributions of different articulatory, acoustic, perceptual, and neural aspects of adaptation in effective communication? To address these questions, we combine behavioral, neural and computational approaches to examine audio-video recordings of live conversations involving interactive tasks to elicit words with specific sound contrasts. Videos of speakers' facial movements during target sound productions and acoustic correlates of the same productions will be analyzed, along with how these differences are perceived. Neural data will also be collected through functional neuroimaging to study the brain activity for both conversation partners simultaneously. Finally, articulatory, acoustic, perceptual, and neural data will be brought together using computational modeling to develop prediction for which attributes improve the likelihood of accurate communication. This research has significant implications in speech science, communication, neuroscience, computer speech technology and speech engineering. Findings will inform how speech adaptation principles can be applied in the development of technologies for adverse listening or visual environments, speech and hearing therapy, language learning and teaching, and human-computer interfaces. We expect this research to contribute to new knowledge leading to clearer channels of communication in Canada's multi-lingual and multimedia society.