Grant: Discovery Grants Program - Individual
Concordia University
$23,000.00
Data Driven Human Motion Synthesis and 3D Reconstruction
EF 2021-22
Résumé
My research program is primarily in three dimensional (3D) graphics. The overarching objective is to efficiently synthesize digital objects and worlds (virtual or augmented reality - VR/AR) and interact with them easily. The current focus is on using sensed data and graphics processing (GPU) for the development of data-driven methods, with application in games, entertainment, engineering and virtual simulation. While earlier work was mainly on algorithmic methods using hand-crafted features, the main idea in the proposal here is to develop new methods which apply machine/deep learning and machine inference to solve 3D graphics problems in 3D reconstruction and human motion. The proposed research has two core research objectives and one technical objective. The first is creating natural looking human action sequences from motion data (for use in games, films and VR/AR). The second is methods for automatic reconstruction of 3D objects and larger scenes from sensed visual data (for use in engineering design, simulation and VR/AR). The technical objective is to build software tools which can empower artists to include sensor driven visual effects in live stage performances, and in interactive documentaries. While there is a lot of research on using deep learning for 3D graphics problems, the proposed work is different and innovative in the way the problems and solutions are formulated. Deep learning solutions depend heavily on data and on extensive experiments to decide on the network architecture and hyper-parameters. Since large annotated data sets are difficult to obtain, my research methodology emphasizes (1) reducing dependence on very large labelled data, yet creating adequate data if needed, (2) initiating research students early into experimentation with appropriate access to infrastructure, (3) formulating problems keeping in mind theoretical foundations as well as eventual interest to industry and artists, and (4) keeping track of state of the art while developing innovative solutions. Specifically, I will investigate combining deep learning and machine inference with hand-crafted features, thereby reducing the need for large data and large neural networks. I will build upon ongoing research with students in my 3D graphics group which has the required infrastructure and industry collaborations. On average 3 PhD and 2 MSc students will be trained per year in leading edge research and technologies of visual data computing. With basic research pursuits being the primary objective, they will work in small teams, as needed, and acquire the skills to implement, test and validate complex systems, and communicate their research results. PhD students, in particular will be trained to develop their own long-term research program in 3D graphics. Basic research will add new knowledge to 3D graphics through publications in relevant venues. New tools will benefit industry, artists, and entertainment and virtual simulation sectors in Canada.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-05729
Name nom
Mudur, Sudhir
Department département
Computer Science and Software Engineering
Organization id
70
Institution établissement
Concordia University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2019
Award amount
23000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2707
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Graphics
Research subject fr
Infographie
Installment
inst:3
Partie
1
Nb partie
5
Application title
Data Driven Human Motion Synthesis and 3D Reconstruction
Keyword
NSERC
Application summary
My research program is primarily in three dimensional (3D) graphics. The overarching objective is to efficiently synthesize digital objects and worlds (virtual or augmented reality - VR/AR) and interact with them easily. The current focus is on using sensed data and graphics processing (GPU) for the development of data-driven methods, with application in games, entertainment, engineering and virtual simulation. While earlier work was mainly on algorithmic methods using hand-crafted features, the main idea in the proposal here is to develop new methods which apply machine/deep learning and machine inference to solve 3D graphics problems in 3D reconstruction and human motion. The proposed research has two core research objectives and one technical objective. The first is creating natural looking human action sequences from motion data (for use in games, films and VR/AR). The second is methods for automatic reconstruction of 3D objects and larger scenes from sensed visual data (for use in engineering design, simulation and VR/AR). The technical objective is to build software tools which can empower artists to include sensor driven visual effects in live stage performances, and in interactive documentaries. While there is a lot of research on using deep learning for 3D graphics problems, the proposed work is different and innovative in the way the problems and solutions are formulated. Deep learning solutions depend heavily on data and on extensive experiments to decide on the network architecture and hyper-parameters. Since large annotated data sets are difficult to obtain, my research methodology emphasizes (1) reducing dependence on very large labelled data, yet creating adequate data if needed, (2) initiating research students early into experimentation with appropriate access to infrastructure, (3) formulating problems keeping in mind theoretical foundations as well as eventual interest to industry and artists, and (4) keeping track of state of the art while developing innovative solutions. Specifically, I will investigate combining deep learning and machine inference with hand-crafted features, thereby reducing the need for large data and large neural networks. I will build upon ongoing research with students in my 3D graphics group which has the required infrastructure and industry collaborations. On average 3 PhD and 2 MSc students will be trained per year in leading edge research and technologies of visual data computing. With basic research pursuits being the primary objective, they will work in small teams, as needed, and acquire the skills to implement, test and validate complex systems, and communicate their research results. PhD students, in particular will be trained to develop their own long-term research program in 3D graphics. Basic research will add new knowledge to 3D graphics through publications in relevant venues. New tools will benefit industry, artists, and entertainment and virtual simulation sectors in Canada.