Grant: Discovery Grants Program - Individual
McMaster University
$29,000.00
Modeling Diverse, Personalized and Expressive Animations for Virtual Characters through Motion Capture, Synthesis and Perception
EF 2022-23
Résumé
Character animation plays a key role in delivering motions for virtual characters in game development, robotics, Virtual Reality (VR) and Augmented Reality (AR) applications. Despite the large effort in modeling motions to be natural and realistic, previous research has been focused on generic motion content learning where stylistic features of how each individual performs the content are largely ignored. Without individual styles, different virtual characters move the same way when the same motion content is needed, which is far from satisfactory to create a diverse virtual world. One major challenge to stylize motions is the lack of large scale motion style databases, and thus no sufficient knowledge of how to effectively model and transfer styles. This proposal focuses on motion style learning, synthesis and transfer. In the long term, our goal is to find ultimate solutions to generate stylized motions with variations that match the diversity in the real-world. The short-term objectives are: we will first establish large scale motion style databases through motion capture technique; from the data, we develop methods to learn effective motion representations and explore generative models to conditionally generate motions with desired styles; we will further discover style transfer models to edit styles while keeping the original motion content. In the five-year period, we will specifically capture, learn and model three stylistic features: demographic styles belonging to different groups of people, e.g. age, gender, and race; personalized styles resulting from personalities, body build for different individuals; expressive styles demonstrating varied emotional and physical states for the same individual under different scenarios. We will set up our databases to cover these style variations, publicize the database for open access, and provide labelling, documentation and technical support to the public. Research findings and source code will be published, solving problems of extracting style features, generating motion styles in a controllable manner, and transferring styles to novel motions. Beyond the five-year term, we will continue adding more styles to our databases, to model a broader picture of motion styles. Our database can directly be used in animating diverse characters in AR/VR and game scenes, it also facilitates other researchers to model motion styles, and stimulates interdisciplinary research in psychology, kinesiology, and art etc. Research findings in motion style modeling supports intelligent applications such as action recognition, style recognition and person identification from motion input. Style synthesis and transfer technology can also be widely applied in the game, entertainment industry, AR/VR applications in education, media, and social networks. By creating virtual characters that authentically embody people in the real world, this work can promote Diversity, Equity and Inclusion of the virtual world.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2022-04920
Name nom
Wang, Yingying
Department département
Computing and Software - Computing and Software
Organization id
27
Institution établissement
McMaster University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2022
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2700
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Information technology
Research subject fr
Technologie de l'information
Installment
inst:1
Partie
1
Nb partie
5
Application title
Modeling Diverse, Personalized and Expressive Animations for Virtual Characters through Motion Capture, Synthesis and Perception
Application summary
Character animation plays a key role in delivering motions for virtual characters in game development, robotics, Virtual Reality (VR) and Augmented Reality (AR) applications. Despite the large effort in modeling motions to be natural and realistic, previous research has been focused on generic motion content learning where stylistic features of how each individual performs the content are largely ignored. Without individual styles, different virtual characters move the same way when the same motion content is needed, which is far from satisfactory to create a diverse virtual world. One major challenge to stylize motions is the lack of large scale motion style databases, and thus no sufficient knowledge of how to effectively model and transfer styles. This proposal focuses on motion style learning, synthesis and transfer. In the long term, our goal is to find ultimate solutions to generate stylized motions with variations that match the diversity in the real-world. The short-term objectives are: we will first establish large scale motion style databases through motion capture technique; from the data, we develop methods to learn effective motion representations and explore generative models to conditionally generate motions with desired styles; we will further discover style transfer models to edit styles while keeping the original motion content. In the five-year period, we will specifically capture, learn and model three stylistic features: demographic styles belonging to different groups of people, e.g. age, gender, and race; personalized styles resulting from personalities, body build for different individuals; expressive styles demonstrating varied emotional and physical states for the same individual under different scenarios. We will set up our databases to cover these style variations, publicize the database for open access, and provide labelling, documentation and technical support to the public. Research findings and source code will be published, solving problems of extracting style features, generating motion styles in a controllable manner, and transferring styles to novel motions. Beyond the five-year term, we will continue adding more styles to our databases, to model a broader picture of motion styles. Our database can directly be used in animating diverse characters in AR/VR and game scenes, it also facilitates other researchers to model motion styles, and stimulates interdisciplinary research in psychology, kinesiology, and art etc. Research findings in motion style modeling supports intelligent applications such as action recognition, style recognition and person identification from motion input. Style synthesis and transfer technology can also be widely applied in the game, entertainment industry, AR/VR applications in education, media, and social networks. By creating virtual characters that authentically embody people in the real world, this work can promote Diversity, Equity and Inclusion of the virtual world.