Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Idea to Innovation

University of Toronto

$14,775.00

A Business / Market Opportunity Assessment for A Speech-Driven Facial Animation System

EF 2019-20

Résumé
For the highly competitive research field of 3D facial animation and speech synchronization, applications such as tv/film, character driven video games, AR/VR and mobile technologies are increasingly moving towards more immersive experiences, across all platforms. This shift places pressure on animators and content creators to innovate with their production pipelines and methodologies to automate animation where possible and to adopt new technologies that allow them to achieve animation results in real-time but with offline hand animated quality. We present in this proposal, a novel speech-driven workflow for expressive, animator-centric facial animation. We show that a complete, realistic human face performance can be automatically generated from an input audio stream and a tagged text transcript, using insights from anatomy, perception, and psycho-linguistic literature. Speech interspersed with directorial tags is the primary interface for animator control over the chronology of facial motion events (blinks, a smile, or an utterance). Animators use the tags in a transcript to annotate the emotional, anatomical, and non-verbal behaviour of a character's face during a vocal performance. An analysis of the audio signal gleans phonetic timing by aligning the tagged transcript with the performance, and it extracts key audio cues to invoke motion events and modulate emotional intensity, speech style, blinks and brow motion. The analyzed output is used to procedurally generate a complete animated face, producing a compact and sparse set of animator-editable motion curves that can interactively drive any typical FACS-based face rig. Our workflow has been used to successfully animate several hours of an upcoming AAA game in eight languages, with little to no need for animator post-editing. This is an application for an I2I Market Assessment grant to reach out to the field and potential customers in order to identify ideal target markets and pain points that will inform and articulate the value proposition and market entry strategy for the successful commercialization of our innovative technology.
Programme
Idea to Innovation
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
538558-2019
Name nom
Singh, Karan
Department département
Computer Science (St. George Campus)
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2019
Award amount
14775
Program id
I2IPJ
Program name en
Idea to Innovation
Program name fr
De l'idée à l'innovation
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2707
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Graphics
Research subject fr
Infographie
Installment
inst:1 - 1
Partie
1
Nb partie
1
Application title
A Business / Market Opportunity Assessment for A Speech-Driven Facial Animation System
Keyword
NSERC
Application summary
For the highly competitive research field of 3D facial animation and speech synchronization, applications such as tv/film, character driven video games, AR/VR and mobile technologies are increasingly moving towards more immersive experiences, across all platforms. This shift places pressure on animators and content creators to innovate with their production pipelines and methodologies to automate animation where possible and to adopt new technologies that allow them to achieve animation results in real-time but with offline hand animated quality. We present in this proposal, a novel speech-driven workflow for expressive, animator-centric facial animation. We show that a complete, realistic human face performance can be automatically generated from an input audio stream and a tagged text transcript, using insights from anatomy, perception, and psycho-linguistic literature. Speech interspersed with directorial tags is the primary interface for animator control over the chronology of facial motion events (blinks, a smile, or an utterance). Animators use the tags in a transcript to annotate the emotional, anatomical, and non-verbal behaviour of a character's face during a vocal performance. An analysis of the audio signal gleans phonetic timing by aligning the tagged transcript with the performance, and it extracts key audio cues to invoke motion events and modulate emotional intensity, speech style, blinks and brow motion. The analyzed output is used to procedurally generate a complete animated face, producing a compact and sparse set of animator-editable motion curves that can interactively drive any typical FACS-based face rig. Our workflow has been used to successfully animate several hours of an upcoming AAA game in eight languages, with little to no need for animator post-editing. This is an application for an I2I Market Assessment grant to reach out to the field and potential customers in order to identify ideal target markets and pain points that will inform and articulate the value proposition and market entry strategy for the successful commercialization of our innovative technology.