Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Collaborative Research and Development Grants

Université Laval

$34,609.00

Deep style transfer for 3D meshes

EF 2019-20

Résumé
Video games, especially large-scale productions such as AAA games, contain thousands of 3D assets that have been crafted by tens or even hundreds of highly skilled artists. In order to give the game a desired artistic look, these artists first model the assets in 3D and assign them default texture. Then, they proceed to carefully ""paint"" additional stylistic details to the texture in order to give them a desired look. This process, commonly known as ""inking"", is tedious, time-consuming and must sometimes be repeated several times when the artistic direction changes. In this grant application, we propose to develop computer vision algorithms for automatically applying a desired style onto the base texture map of 3D objects, and as such reproduce the inking process done by artists. The algorithms must be automatic, should be able to learn the stylization automatically from examples, and should be generic enough to be applied to thousands of different 3D models. Our key idea is to rely on recent image-to-image translation approaches based on deep generative adversarial networks (GANs). To attain these objectives, we will first build a dataset of 3D objects and their stylized texture maps based on the database acquired by the industrial partner over the years. Then, we will experiemnt with image-to-image translation GANs applied to texture maps, and to 3D renders of the object. Finally, we will compare both approaches, and explore future directions to reduce the need for large training sets. This application will contribute to the video game industry in Canada, which contributed $3.7B in GDP in 2017. The partnership between U. Laval and Gearbox Studio Québec will both exploit the strengths in deep learning and computer vision of the research team at U. Laval and the expertise in video game development at Gearbox. In particular, the industrial partner possesses a unique dataset of thousands of stylized 3D models used in past game productions which will be leveraged in this work.
Programme
Collaborative Research and Development Grants
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
537961-2018
Name nom
Lalonde, JeanFrançois
Department département
Génie électrique et génie informatique
Organization id
51
Institution établissement
Université Laval
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2018
Award amount
34609
Program id
CRDPJ
Program name en
Collaborative Research and Development Grants
Program name fr
Subventions de recherche et développement coopérative
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Installment
inst:1 - 1
Partie
1
Nb partie
1
Application title
Deep style transfer for 3D meshes
Keyword
NSERC
Application summary
Video games, especially large-scale productions such as AAA games, contain thousands of 3D assets that have been crafted by tens or even hundreds of highly skilled artists. In order to give the game a desired artistic look, these artists first model the assets in 3D and assign them default texture. Then, they proceed to carefully ""paint"" additional stylistic details to the texture in order to give them a desired look. This process, commonly known as ""inking"", is tedious, time-consuming and must sometimes be repeated several times when the artistic direction changes. In this grant application, we propose to develop computer vision algorithms for automatically applying a desired style onto the base texture map of 3D objects, and as such reproduce the inking process done by artists. The algorithms must be automatic, should be able to learn the stylization automatically from examples, and should be generic enough to be applied to thousands of different 3D models. Our key idea is to rely on recent image-to-image translation approaches based on deep generative adversarial networks (GANs). To attain these objectives, we will first build a dataset of 3D objects and their stylized texture maps based on the database acquired by the industrial partner over the years. Then, we will experiemnt with image-to-image translation GANs applied to texture maps, and to 3D renders of the object. Finally, we will compare both approaches, and explore future directions to reduce the need for large training sets. This application will contribute to the video game industry in Canada, which contributed $3.7B in GDP in 2017. The partnership between U. Laval and Gearbox Studio Québec will both exploit the strengths in deep learning and computer vision of the research team at U. Laval and the expertise in video game development at Gearbox. In particular, the industrial partner possesses a unique dataset of thousands of stylized 3D models used in past game productions which will be leveraged in this work.