Grant: Discovery Grants Program - Individual
École Polytechnique de Montréal
$29,000.00
Generative models for 3D textures
EF 2024-25
Résumé
3D textures are the volumetric equivalent of 2D raster textures. They offer several advantages over their 2D counterparts. However, despite recent progress and unlike 2D textures, dedicated techniques that allow to model and create such textures are limited. Unlike scanning a 2D image, digitization of volumetric color information is usually impractical. This creates the need for efficient and controllable synthesis algorithms. One reason is the lack of intuitive specification models allowing a user to indicate what he has in mind. Another reason is the lack of scalable models that can handle three dimensions at high resolution. Our main research objective is to pursue fundamental and applied research for the development of novel representations and design algorithms of 3D textures. We target four specific objectives. 1) User centered design models for controlling 3D textures synthesis. First, we will offer a highly controllable synthesis method. This raises a major issue, especially when tackling textures with distinguishable patterns. Unlike 2D synthesis, where the user can provide a small example to input the algorithm, providing 3D examples is impractical. We will propose novel controls using a conditionnal adversarial network. 2) Structural sound models for enriching 3D printing. 3D printing offers a challenging context allowing designers and artists with an exciting opportunity to imagine and produce complex patterns and objects. We will propose a method that tackles structural soundness for printed 3D textures based objects. 3) Temporal coherent models for video style transferring. This is the task of automatically combining the content of one video and the style of an image into a stylized output video. Typically, the style of an artistic image is transferred to a photo-realistic video. We propose to use a 3D convolutional neural network to stylize local video blocks, several frames at a time. At the texture scale, this will guarantee consistency in both spatial and temporal dimensions, avoiding the estimation of the optical flow. 4) Edge--aware models for data- augmentation in medical applications. Deep learning techniques have been successful in medical image analysis. However, large labeled training databases are needed to achieve good performance. Such data is usually difficult to gather since it usually needs manual labeling. We will evaluate how data augmentation through 3D constrained texture synthesis can improve performances. We will focus on modalities relying on texture features such as histology and MRI. The proposed research program will develop tools and techniques enabling breakthrough innovation, leading to major advances in several applications, related to the Canadian computer animation and video games industry, and the medical field.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2021-03271
Name nom
Hurtut, Thomas
Department département
Génie informatique
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2021
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2707
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Graphics
Research subject fr
Infographie
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Generative models for 3D textures
Keywords
3d textures; texture synthesis; style transferring; data-augmentation; texture descriptor; texture authoring; texture models
Application summary
3D textures are the volumetric equivalent of 2D raster textures. They offer several advantages over their 2D counterparts. However, despite recent progress and unlike 2D textures, dedicated techniques that allow to model and create such textures are limited. Unlike scanning a 2D image, digitization of volumetric color information is usually impractical. This creates the need for efficient and controllable synthesis algorithms. One reason is the lack of intuitive specification models allowing a user to indicate what he has in mind. Another reason is the lack of scalable models that can handle three dimensions at high resolution. Our main research objective is to pursue fundamental and applied research for the development of novel representations and design algorithms of 3D textures. We target four specific objectives. 1) User centered design models for controlling 3D textures synthesis. First, we will offer a highly controllable synthesis method. This raises a major issue, especially when tackling textures with distinguishable patterns. Unlike 2D synthesis, where the user can provide a small example to input the algorithm, providing 3D examples is impractical. We will propose novel controls using a conditionnal adversarial network. 2) Structural sound models for enriching 3D printing. 3D printing offers a challenging context allowing designers and artists with an exciting opportunity to imagine and produce complex patterns and objects. We will propose a method that tackles structural soundness for printed 3D textures based objects. 3) Temporal coherent models for video style transferring. This is the task of automatically combining the content of one video and the style of an image into a stylized output video. Typically, the style of an artistic image is transferred to a photo-realistic video. We propose to use a 3D convolutional neural network to stylize local video blocks, several frames at a time. At the texture scale, this will guarantee consistency in both spatial and temporal dimensions, avoiding the estimation of the optical flow. 4) Edge--aware models for data- augmentation in medical applications. Deep learning techniques have been successful in medical image analysis. However, large labeled training databases are needed to achieve good performance. Such data is usually difficult to gather since it usually needs manual labeling. We will evaluate how data augmentation through 3D constrained texture synthesis can improve performances. We will focus on modalities relying on texture features such as histology and MRI. The proposed research program will develop tools and techniques enabling breakthrough innovation, leading to major advances in several applications, related to the Canadian computer animation and video games industry, and the medical field.