Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of British Columbia

$26,000.00

3D Articulatory Speech Synthesis using Biomechanical Models of the Oral, Pharyngeal and Laryngeal Complex

EF 2014-15

Résumé
For years, traditional formant-based and acoustic-based speech synthesis techniques have largely overshadowed articulatory synthesis research. While successful in some domains, they still cannot produce natural sounding speech from text. Articulatory speech synthesis, in contrast, continues to progress steadily at the fringes of both industrial and academic interest and is now poised to provide the necessary platform to overcome basic problems in speech processing and, I believe, represents the next major advance in speech synthesis technology. As part of my long-term research program, I have been establishing computer modeling and simulation of the oral, pharyngeal and laryngeal (OPAL) complex as a critical and central tool in many disciplines, including engineering, linguistics, education, entertainment, anatomy, physiology and medicine. From this effort, the infrastructure necessary to make large advances in articulatory speech synthesis is now available. Thus, I propose to develop the OPAL complex models to include aero-acoustic simulation to create the next generation of validated, articulatory speech synthesis. This research builds upon the detailed biomechanical models of the human of the upper airway that have been used to study the mechanics of speech behavior, swallowing and mastication along with the tools needed for fast, accurate coupled FEM and rigid body simulations. The main research activities are: 1. Integration of model geometries and biomechanics to create a complete biomechanical model of the OPAL complex capable of articulatory speech synthesis; 2. Improvement of the existing 1D Navier Stokes approach to provide improved coupling between laryngeal excitation of the vocal tract filter by leveraging the work of [Birkholz et al, 2006]; 3. Development of 3D aero-acoustic simulation of vocal tract noise sources based on fluid simulation as well as CFD based approximations; 4. Creation of inverse methods for driving speech articulation by extending the work of [Stavness et al, 2013] to include support for propagating both kinematic targets from image data or electromagnetic articulography (EMA) of speech and acoustic targets, backward through the aeroacoustic model, to derive muscle activations based on inverse dynamics techniques. Speech articulation constraints will be used to help make the system better determined. 5. Creation of advanced coupled vocal fold models that will oscillate under tension and airflow. We are poised to make a substantial advance in articulatory speech synthesis. The last decade of effort in creating the medical imaging techniques, speech measurements, complex 3D biomechanical models of the OPAL complex and tools for fast, accurate coupled FEM/rigid body simulation provides the necessary infrastructure for this project. Canada stands to benefit from being at the forefront of speech synthesis technology that support the next wave of applications. Including articulatory speech synthesis into our open source toolkit will support international efforts in speech research contributing profound new knowledge from a deeper understanding of human speech production.
Programme
Discovery Grants Program - Individual
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2014-05949
Name nom
Fels, Sidney
Department département
Electrical and Computer Engineering
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2014
Competition year année de concours
2014
Award amount
26000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2716
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Virtual reality and related simulations
Research subject fr
Réalité virtuelle et simulations connexes
Partie
1
Nb partie
1
Application title
3D Articulatory Speech Synthesis using Biomechanical Models of the Oral, Pharyngeal and Laryngeal Complex
Keyword
NSERC
Application summary
For years, traditional formant-based and acoustic-based speech synthesis techniques have largely overshadowed articulatory synthesis research. While successful in some domains, they still cannot produce natural sounding speech from text. Articulatory speech synthesis, in contrast, continues to progress steadily at the fringes of both industrial and academic interest and is now poised to provide the necessary platform to overcome basic problems in speech processing and, I believe, represents the next major advance in speech synthesis technology. As part of my long-term research program, I have been establishing computer modeling and simulation of the oral, pharyngeal and laryngeal (OPAL) complex as a critical and central tool in many disciplines, including engineering, linguistics, education, entertainment, anatomy, physiology and medicine. From this effort, the infrastructure necessary to make large advances in articulatory speech synthesis is now available. Thus, I propose to develop the OPAL complex models to include aero-acoustic simulation to create the next generation of validated, articulatory speech synthesis. This research builds upon the detailed biomechanical models of the human of the upper airway that have been used to study the mechanics of speech behavior, swallowing and mastication along with the tools needed for fast, accurate coupled FEM and rigid body simulations. The main research activities are: 1. Integration of model geometries and biomechanics to create a complete biomechanical model of the OPAL complex capable of articulatory speech synthesis; 2. Improvement of the existing 1D Navier Stokes approach to provide improved coupling between laryngeal excitation of the vocal tract filter by leveraging the work of [Birkholz et al, 2006]; 3. Development of 3D aero-acoustic simulation of vocal tract noise sources based on fluid simulation as well as CFD based approximations; 4. Creation of inverse methods for driving speech articulation by extending the work of [Stavness et al, 2013] to include support for propagating both kinematic targets from image data or electromagnetic articulography (EMA) of speech and acoustic targets, backward through the aeroacoustic model, to derive muscle activations based on inverse dynamics techniques. Speech articulation constraints will be used to help make the system better determined. 5. Creation of advanced coupled vocal fold models that will oscillate under tension and airflow. We are poised to make a substantial advance in articulatory speech synthesis. The last decade of effort in creating the medical imaging techniques, speech measurements, complex 3D biomechanical models of the OPAL complex and tools for fast, accurate coupled FEM/rigid body simulation provides the necessary infrastructure for this project. Canada stands to benefit from being at the forefront of speech synthesis technology that support the next wave of applications. Including articulatory speech synthesis into our open source toolkit will support international efforts in speech research contributing profound new knowledge from a deeper understanding of human speech production.