Grant: Discovery Grants Program - Individual
École de technologie supérieure
$25,000.00
Scalable, robust, and efficient forward and differentiable rendering
EF 2024-25
Résumé
Photorealistic image synthesis is essential to many applications in feature film production, video game development, architectural design, and medical imaging. Images are generated via a forward rendering process that simulates, using physics-based mathematical models, how light propagates and scatters in a virtual 3D scene. Conversely, inverse rendering performs similar simulations but attempts to recover the object's parameters in a virtual environment from a real (or synthetic) image only. This direct inversion is often not possible due to the complex radiative coupling between different objects of the scenes. Differentiable rendering allows to compute and measure how each object's parameters, such as its shape and its material, impact the final generated image. This information is then used to guide an optimization process that will update the objects' parameters iteratively until the target image is produced.
The long-standing challenge in these rendering algorithms is the efficient evaluation of light transport. Indeed, there is no direct solution to calculate the light distribution in a 3D environment. For this reason, we are forced to rely on numerical statistical methods to approximate these radiometric quantities. However, these methods generate errors in estimated values, which manifest themselves as noise in the synthetic images or in the optimized object parameters.
The objective of this research plan is to address this noise issue by developing new sampling methods that can estimate light transport more accurately, robustly, and efficiently. These improvements are critical for renderers that must deliver images at interactive framerates, that do not require tuning user parameters, or that are robust enough to be deployed in the wild to be used by artists. This proposal will target two medium-term objectives to achieve significant advancement toward this long-term objective. First, we will develop robust operators that transform the structure from previously generated paths. These new operators will automatically adapt their sampling decisions based on available data extracted from the ongoing light transport simulation. These operators will then be generalized to develop new adaptive sampling techniques in key forward rendering applications. The second objective will be to study and develop tailored sampling strategies to scale current differentiable rendering frameworks.
This program will bring important benefits to Canada for three reasons. First, the students trained through this research program will be experts in light transport, expertise that is needed for the Canadian computer graphics and visual effects industry. Second, our advances in differentiable rendering will allow it to become a central technique for a new automatic way of content creation and design. Last, the methods developed can also be adapted to other transport problems, such as sound simulation, neutron transport, or geometry processing.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2022-03182
Name nom
Gruson, Adrien
Department département
Département de LOGTI
Organization id
53
Institution établissement
École de technologie supérieure
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2022
Award amount
25000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2707
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Graphics
Research subject fr
Infographie
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Scalable, robust, and efficient forward and differentiable rendering
Keywords
global illumination; light transport simulation; differentiable rendering; inverse rendering; ray tracing; importance sampling
Application summary
Photorealistic image synthesis is essential to many applications in feature film production, video game development, architectural design, and medical imaging. Images are generated via a forward rendering process that simulates, using physics-based mathematical models, how light propagates and scatters in a virtual 3D scene. Conversely, inverse rendering performs similar simulations but attempts to recover the object's parameters in a virtual environment from a real (or synthetic) image only. This direct inversion is often not possible due to the complex radiative coupling between different objects of the scenes. Differentiable rendering allows to compute and measure how each object's parameters, such as its shape and its material, impact the final generated image. This information is then used to guide an optimization process that will update the objects' parameters iteratively until the target image is produced.
The long-standing challenge in these rendering algorithms is the efficient evaluation of light transport. Indeed, there is no direct solution to calculate the light distribution in a 3D environment. For this reason, we are forced to rely on numerical statistical methods to approximate these radiometric quantities. However, these methods generate errors in estimated values, which manifest themselves as noise in the synthetic images or in the optimized object parameters.
The objective of this research plan is to address this noise issue by developing new sampling methods that can estimate light transport more accurately, robustly, and efficiently. These improvements are critical for renderers that must deliver images at interactive framerates, that do not require tuning user parameters, or that are robust enough to be deployed in the wild to be used by artists. This proposal will target two medium-term objectives to achieve significant advancement toward this long-term objective. First, we will develop robust operators that transform the structure from previously generated paths. These new operators will automatically adapt their sampling decisions based on available data extracted from the ongoing light transport simulation. These operators will then be generalized to develop new adaptive sampling techniques in key forward rendering applications. The second objective will be to study and develop tailored sampling strategies to scale current differentiable rendering frameworks.
This program will bring important benefits to Canada for three reasons. First, the students trained through this research program will be experts in light transport, expertise that is needed for the Canadian computer graphics and visual effects industry. Second, our advances in differentiable rendering will allow it to become a central technique for a new automatic way of content creation and design. Last, the methods developed can also be adapted to other transport problems, such as sound simulation, neutron transport, or geometry processing.