Grant: Alliance Grants
Université Laval
$24,350.00
Learning to light and relight images
FY 2023-24
Summary
Augmented reality is the process of combining the virtual with the real. For the combination to look realistic, the virtual content must match the real: their orientation, lighting, and presence in the scene must all match the real-world conditions. Achieving highly realistic effects is possible (as demonstrated on a regular basis in blockbuster movies), but doing so requires tremendous amounts of manpower---indeed, all special effects nowadays require tens, if not hundreds, of skilled artists painstakingly working on each shot. In this proposal, we aim at taking a step towards automating the process of realistically inserting virtual objects into real photographs. Specifically, we address the problem of lighting estimation along the following two main axes. First, we will demonstrate that the full, spatially-varying lighting conditions of a scene can be robustly estimated from either a single, or multiple images of that scene. Second, we will show how images of objects and even full scenes can be interactively relit according to a target lighting.
Program
Alliance Grants
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
557208-2020
Name nom
Lalonde, Jean-François J-F
Department département
Génie électrique et génie informatique
Organization id
51
Institution établissement
Université Laval
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2020
Award amount
24350
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2605
Research subject group en
Robotics
Research subject group fr
Pattern analysis and machine intelligence
Research subject en
Robotique
Research subject fr
Analyse de formes et intelligence artificielle
Application title
Learning to light and relight images
Keyword
computer graphics; computer vision; deep learning; high dynamic range; image relighting; lighting capture; lighting estimation; machine learning; neural rendering
Application summary
Augmented reality is the process of combining the virtual with the real. For the combination to look realistic, the virtual content must match the real: their orientation, lighting, and presence in the scene must all match the real-world conditions. Achieving highly realistic effects is possible (as demonstrated on a regular basis in blockbuster movies), but doing so requires tremendous amounts of manpower---indeed, all special effects nowadays require tens, if not hundreds, of skilled artists painstakingly working on each shot. In this proposal, we aim at taking a step towards automating the process of realistically inserting virtual objects into real photographs. Specifically, we address the problem of lighting estimation along the following two main axes. First, we will demonstrate that the full, spatially-varying lighting conditions of a scene can be robustly estimated from either a single, or multiple images of that scene. Second, we will show how images of objects and even full scenes can be interactively relit according to a target lighting.