Grant: Discovery Grants Program - Individual
École de technologie supérieure
$23,000.00
Real-time synthesis and capture of contact rich human interactions
FY 2023-24
Summary
INTRODUCTIONMotion synthesis for virtual characters involves computing rotations and joint positions for an articulated three-dimensional (3D) structure, or skeleton. Examples of this are ubiquitous in modern computer graphics applications, such as video games, visual effects, and virtual reality (VR). Large strides have been made in recent years on the automatic synthesis of motion for characters, yet methods still fail to achieve the level of skill and physical interactivity of real humans. Motions such as climbing, tumbling, and dexterous manipulation are notoriously difficult to synthesize; tracking and reconstructing these motions is similarly difficult.Contact is an important factor in these settings and must be given special consideration. Physical models can be used to improve realism, but robustness and computational performance become concerns due to the non-linearities and overhead associated with complex physical simulations. Automatic synthesis methods for physical characters are therefore limited in their scope, and as such video game and VR developers have been slow to adopt them.OBJECTIVEMy research focuses on motion synthesis of virtual characters where the motion demands significant amounts of physical interaction, or contact. There will be a special effort to develop methods for real-time applications with a limited computational budget due to strict framerate requirements. The techniques I develop over the next five years will give rise to new ways for skilled motion synthesis for 3D characters, offering unprecedented interactivity that rivals the fidelity of real humans. For this purpose, I will pursue three parallel, but complementary, tracks:1. Fast and stable physical simulations involving contact;2. Datasets and data-driven methods for complex physical interaction;3. Robust motion synthesis for virtual characters.The anticipated results of my research will be innovative methods for interactive character animation, motion planning, physics simulation, and motion reconstruction. Solutions uncovered along the way will have immediate practical implications. This will result not only in novel technology for character animation in games, but will also impact robotics simulation, VR training, and performance capture applications. The long-term impact will be a multiplication of progress across all tracks.SCIENTIFIC APPROACHMy general scientific approach is to augment physical simulations with data-driven methods where appropriate. This includes collecting real-world examples to build parameterized models that encapsulate natural phenomena. Such models may then be used to guide an optimization involving physics simulation. Since my research focuses on real-time applications, model reduction techniques are also an essential component. Challenges lie in approximating physical behaviour without compromising overall fidelity.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2018-05723
Name nom
Andrews, Sheldon
Department département
Software and IT Engineering
Organization id
53
Institution établissement
École de technologie supérieure
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2018
Award amount
23000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
800
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information and communication services
Area of application fr
Services d'information et de télécommunications
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Real-time synthesis and capture of contact rich human interactions
Keyword
Character Animation; Computer graphics; Data-driven methods; Human motion synthesis; Motion capture; Physics simulation
Application summary
INTRODUCTIONMotion synthesis for virtual characters involves computing rotations and joint positions for an articulated three-dimensional (3D) structure, or skeleton. Examples of this are ubiquitous in modern computer graphics applications, such as video games, visual effects, and virtual reality (VR). Large strides have been made in recent years on the automatic synthesis of motion for characters, yet methods still fail to achieve the level of skill and physical interactivity of real humans. Motions such as climbing, tumbling, and dexterous manipulation are notoriously difficult to synthesize; tracking and reconstructing these motions is similarly difficult.Contact is an important factor in these settings and must be given special consideration. Physical models can be used to improve realism, but robustness and computational performance become concerns due to the non-linearities and overhead associated with complex physical simulations. Automatic synthesis methods for physical characters are therefore limited in their scope, and as such video game and VR developers have been slow to adopt them.OBJECTIVEMy research focuses on motion synthesis of virtual characters where the motion demands significant amounts of physical interaction, or contact. There will be a special effort to develop methods for real-time applications with a limited computational budget due to strict framerate requirements. The techniques I develop over the next five years will give rise to new ways for skilled motion synthesis for 3D characters, offering unprecedented interactivity that rivals the fidelity of real humans. For this purpose, I will pursue three parallel, but complementary, tracks:1. Fast and stable physical simulations involving contact;2. Datasets and data-driven methods for complex physical interaction;3. Robust motion synthesis for virtual characters.The anticipated results of my research will be innovative methods for interactive character animation, motion planning, physics simulation, and motion reconstruction. Solutions uncovered along the way will have immediate practical implications. This will result not only in novel technology for character animation in games, but will also impact robotics simulation, VR training, and performance capture applications. The long-term impact will be a multiplication of progress across all tracks.SCIENTIFIC APPROACHMy general scientific approach is to augment physical simulations with data-driven methods where appropriate. This includes collecting real-world examples to build parameterized models that encapsulate natural phenomena. Such models may then be used to guide an optimization involving physics simulation. Since my research focuses on real-time applications, model reduction techniques are also an essential component. Challenges lie in approximating physical behaviour without compromising overall fidelity.