Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

Simon Fraser University

$25,000.00

Generating Usable 3D Objects via Deep Learning

FY 2024-25

Summary
Deep learning has recently been successful in 3D geometric modelling and computer vision. Various applications of deep learning exist that generate or work with 3D shapes (e.g., shape generation, 3D reconstruction, etc.). Until now, 3D generated shapes are usually simple and fragile, lack geometric features, possess noise or irregularities, and are not accompanied by high-quality textures. To utilize the result of generative models in animation or manufacturing industries, we need high-quality, durable, and textured shapes. This proposal aims to design deep generative models that can produce "usable" 3D shapes meaning that they are fabricable, realistic (i.e., detailed and textured), diverse, and functional. While choosing appropriate deep generative models might be problem specific (e.g., convolutional vs implicit), deep implicit models have been successful to represent a shape as they provide a smooth representation and can handle topological varieties. Therefore, the main focus of this proposal is to develop methodologies to advance current deep implicit models to produce "usable" shapes. My ultimate target would be to design generative models capable of producing shapes with qualities that one cannot distinguish from those professionally made by an artist or a modeler. To achieve these goals, I start by defining four projects to explore different aspects of "shape usability" and its applications. First, I intend to enhance deep generative models to produce "fabricable" shapes that can be fabricated using available devices such as CNC machines or 3D prints. Depending on the fabrication type, a shape should hold certain properties such as "carvability" or "balance" for an efficient fabrication. Second project is to generate detailed shapes with appropriate textures by learning a mapping between texture domain and geometry. This is important as the utilized shapes in industry (e.g., video games) are often accompanied by complex textures. Third project is to add diversity to the generated shapes by diversifying style or functionality. To do so, shapes should be generated with respect to the whole data and mode collapse should be avoided. Also, style or functionality should be disentangled from the shape's content to transfer them across models. Lastly, I would like to collect and synthesize useful 3D datasets and explore the applications of generative models for data augmentation. For instance, capable generative models can be used to augment driving scenes by synthesizing rare scenes to train autonomous cars that behave safely in complex driving scenarios. Previous works on images have shown that generative models are capable of producing super-realistic images (e.g., StyleGAN). Also, recent advancements on 3D generative models especially deep implicit networks have shown a promising future. This proposal is an attempt to add new knowledge to the area of 3D generative models and advance them to produce more realistic and usable 3D shapes.
Program
Discovery Grants Program - Individual
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2022-03111
Name nom
Mahdavi-Amiri, Ali
Department département
Computing Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2022
Award amount
25000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Generating Usable 3D Objects via Deep Learning
Keywords
deep learning; computational fabrication; geometric modeling; data augmentation; shape processing
Application summary
Deep learning has recently been successful in 3D geometric modelling and computer vision. Various applications of deep learning exist that generate or work with 3D shapes (e.g., shape generation, 3D reconstruction, etc.). Until now, 3D generated shapes are usually simple and fragile, lack geometric features, possess noise or irregularities, and are not accompanied by high-quality textures. To utilize the result of generative models in animation or manufacturing industries, we need high-quality, durable, and textured shapes. This proposal aims to design deep generative models that can produce "usable" 3D shapes meaning that they are fabricable, realistic (i.e., detailed and textured), diverse, and functional. While choosing appropriate deep generative models might be problem specific (e.g., convolutional vs implicit), deep implicit models have been successful to represent a shape as they provide a smooth representation and can handle topological varieties. Therefore, the main focus of this proposal is to develop methodologies to advance current deep implicit models to produce "usable" shapes. My ultimate target would be to design generative models capable of producing shapes with qualities that one cannot distinguish from those professionally made by an artist or a modeler. To achieve these goals, I start by defining four projects to explore different aspects of "shape usability" and its applications. First, I intend to enhance deep generative models to produce "fabricable" shapes that can be fabricated using available devices such as CNC machines or 3D prints. Depending on the fabrication type, a shape should hold certain properties such as "carvability" or "balance" for an efficient fabrication. Second project is to generate detailed shapes with appropriate textures by learning a mapping between texture domain and geometry. This is important as the utilized shapes in industry (e.g., video games) are often accompanied by complex textures. Third project is to add diversity to the generated shapes by diversifying style or functionality. To do so, shapes should be generated with respect to the whole data and mode collapse should be avoided. Also, style or functionality should be disentangled from the shape's content to transfer them across models. Lastly, I would like to collect and synthesize useful 3D datasets and explore the applications of generative models for data augmentation. For instance, capable generative models can be used to augment driving scenes by synthesizing rare scenes to train autonomous cars that behave safely in complex driving scenarios. Previous works on images have shown that generative models are capable of producing super-realistic images (e.g., StyleGAN). Also, recent advancements on 3D generative models especially deep implicit networks have shown a promising future. This proposal is an attempt to add new knowledge to the area of 3D generative models and advance them to produce more realistic and usable 3D shapes.