Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Simon Fraser University

$36,000.00

Large Hybrid Animation Models

EF 2024-25

Résumé
Character animation plays an essential role in creating believable virtual characters for various applications, such as digital humans and animals for film making and metaverse platforms, crowd behavior modeling and simulation, human-computer and human-robot interactions, and video games and serious training software. The motion intelligence of existing animation models, however, still cannot match that of real human performers, in terms of the quality and heterogeneity of the synthesized motions, and the ability to plan and achieve complex motor tasks. Our goal is to develop 3D character animation models that can pass the Turing test and produce motions indistinguishable from captured human movements and activities. Our key idea is to learn large character animation models from publicly available datasets. We draw our inspiration from the record-breaking success of large language models (LLM), such as GPT-4, that have revolutionized how we learn, work, and communicate. Such powerful models further enable realistic and appealing artificial intelligence-generated contents (AIGC), such as images synthesized by stable diffusion. The success of large pretrained neural network models requires Internet-scale training datasets though. For example, GPT-4 has 45 gigabytes of training data, and stable diffusion used 2.3 billion images initially. In contrast to the vast amount of text, image, or video data available on the Internet, the largest publicly available animation dataset is about 40 hours, due to inherent difficulties of authoring or capturing animation data. We see physics-based animation methods as potential solutions to such a hunger for data. First, physics-based models can enhance reference data by reducing the noise and artifacts within low-quality data. Second, physics-based models can also learn skills from sparse demonstrations and generalize the learnt skills to different environments with new constraints in multiple styles. Lastly, physics-based models, such as deep reinforcement learning methods, can discover new motor skills without any example, purely from interactions with the environment and following physics constraints. To summarize, we wish to train powerful large character animation models, by augmenting kinematic animation data with physics-based skill learning and control. We expect large scientific, economic, and social impacts from the proposed research, as modeling of human movements is a fundamental problem in computer animation, computer vision, mixed reality, biomechanics, and humanoid robotics. The results of this research are directly transferable to animation tools and systems for the movie and game industries, as well as virtual reality applications and metaverse platforms for social networking, commerce, entertainment, and communication. In addition, our research will enable non-experts to create animation content in easy and intuitive ways as we produce texts, images, and videos today.
Programme
Discovery Grants Program - Individual
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2024-06752
Name nom
Yin, KangKang
Department département
Computing Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
36000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2717
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Modelling and simulation studies
Research subject fr
Études de modélisation et de simulation
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Large Hybrid Animation Models
Keywords
character animation; generative models; kinematic animation; physics-based animation; deep learning; deep reinforcement learning; human movement modeling
Application summary
Character animation plays an essential role in creating believable virtual characters for various applications, such as digital humans and animals for film making and metaverse platforms, crowd behavior modeling and simulation, human-computer and human-robot interactions, and video games and serious training software. The motion intelligence of existing animation models, however, still cannot match that of real human performers, in terms of the quality and heterogeneity of the synthesized motions, and the ability to plan and achieve complex motor tasks. Our goal is to develop 3D character animation models that can pass the Turing test and produce motions indistinguishable from captured human movements and activities. Our key idea is to learn large character animation models from publicly available datasets. We draw our inspiration from the record-breaking success of large language models (LLM), such as GPT-4, that have revolutionized how we learn, work, and communicate. Such powerful models further enable realistic and appealing artificial intelligence-generated contents (AIGC), such as images synthesized by stable diffusion. The success of large pretrained neural network models requires Internet-scale training datasets though. For example, GPT-4 has 45 gigabytes of training data, and stable diffusion used 2.3 billion images initially. In contrast to the vast amount of text, image, or video data available on the Internet, the largest publicly available animation dataset is about 40 hours, due to inherent difficulties of authoring or capturing animation data. We see physics-based animation methods as potential solutions to such a hunger for data. First, physics-based models can enhance reference data by reducing the noise and artifacts within low-quality data. Second, physics-based models can also learn skills from sparse demonstrations and generalize the learnt skills to different environments with new constraints in multiple styles. Lastly, physics-based models, such as deep reinforcement learning methods, can discover new motor skills without any example, purely from interactions with the environment and following physics constraints. To summarize, we wish to train powerful large character animation models, by augmenting kinematic animation data with physics-based skill learning and control. We expect large scientific, economic, and social impacts from the proposed research, as modeling of human movements is a fundamental problem in computer animation, computer vision, mixed reality, biomechanics, and humanoid robotics. The results of this research are directly transferable to animation tools and systems for the movie and game industries, as well as virtual reality applications and metaverse platforms for social networking, commerce, entertainment, and communication. In addition, our research will enable non-experts to create animation content in easy and intuitive ways as we produce texts, images, and videos today.