Grant: Alliance Grants
The University of British Columbia
$55,456.00
Space-time 4D Reconstruction with Video Generative Models
FY 2024-25
Summary
Generative Artificial Intelligence (GenAI) has shown incredible potential, not only scientifically, but also to the wider public through various Large Language Models (LLM) represented by ChatGPT and AI-generated images. More recently, GenAI has also enabled the generation of video content, that is indistinguishable from real-world videos actually taken by people. All of these, however, are all within your monitors or screens---they are images, videos, text, or even audio, but they are all within your device.
In this research project, we will go beyond your device. We will innovate GenAI methods and utilize them to reconstruct the world in 4D from limited data, at a limit as limited as a mere few lines of text. We will then create these 4D representations in a way that users can re-live them through various Virtual Reality devices. We will further allow these techniques to be as computationally efficient as possible, such that they do not have to run on massive compute clusters that the general public cannot afford, thus making the technology accessible to everyone.
Program
Alliance Grants
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
597812-2024
Name nom
Yi, Kwang Moo KM
Department département
Computer Science
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
55456
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Space-time 4D Reconstruction with Video Generative Models
Keywords
3d computer vision; 3d reconstruction; diffusion models; dynamic 3d reconstruction; generative models; neural rendering; text-to-3d; text-to-4d; text-to-video; video diffusion models
Application summary
Generative Artificial Intelligence (GenAI) has shown incredible potential, not only scientifically, but also to the wider public through various Large Language Models (LLM) represented by ChatGPT and AI-generated images. More recently, GenAI has also enabled the generation of video content, that is indistinguishable from real-world videos actually taken by people. All of these, however, are all within your monitors or screens---they are images, videos, text, or even audio, but they are all within your device.
In this research project, we will go beyond your device. We will innovate GenAI methods and utilize them to reconstruct the world in 4D from limited data, at a limit as limited as a mere few lines of text. We will then create these 4D representations in a way that users can re-live them through various Virtual Reality devices. We will further allow these techniques to be as computationally efficient as possible, such that they do not have to run on massive compute clusters that the general public cannot afford, thus making the technology accessible to everyone.