Grant: Discovery Grants Program - Individual
University of Toronto
$80,000.00
Computational Sensing for Extreme Computer Vision
EF 2024-25
Résumé
The main goal of my research program over the next five years will be to study and develop novel computational cameras and sensing techniques that confer extreme vision capabilities to the systems employing them. My work aims to push dynamic 2D and 3D imaging to its limits along multiple axes---3D accuracy, low-light performance, speed, dynamic range, robustness, non-line-of-sight imaging ability, etc---by exploiting two emerging families of computational cameras:
The first---time-resolved SPADs---have pixels sensitive enough to detect individual photons; they are used widely for depth imaging, biological imaging, and looking around corners, and their pixels output a never-ending stream of highly-precise timestamps for the photons they detect. This stream is usually discarded as too massive to process but our initial investigations suggest that the information contained in the timestamp streams makes these cameras extemely powerful: able to measure distances down to microns when combined with a laser, and able to passively record video at speeds almost a billion faster than conventional video cameras.
The second---cameras with pixelwise-programmable exposure---was pioneered by an interdisciplinary team that I built at the University of Toronto, with a vision to provide a "swiss-army-knife sensor" for high-performance 2D and 3D imaging. Now, after more than seven years of pushing the state of the art in speed, pixel size and resolution with new CMOS sensor designs and new camera prototypes, we finally have programmable VGA-resolution sensors with one-of-a-kind capabilities for high-speed, robust 2D and 3D vision.
We seek to unlock the potential of these cameras by (1) laying the foundations for 2D and 3D computer vision at the photon limit, (2) using the flexibility offered by our programmable cameras to solve hard visual inference tasks and (3) integrating sensing, optics and computation even further, to expand our imaging systems' capabilities.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2023-05609
Name nom
Kutulakos, Kiriakos
Department département
Computer Science
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
80000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Computational Sensing for Extreme Computer Vision
Keywords
computational cameras; computational sensors; single-photon imaging; light detection and ranging; high-speed imaging; 3d shape acquisition; extreme computer vision; high-dynamic range imaging; adaptive imaging; computer vision in the dark
Application summary
The main goal of my research program over the next five years will be to study and develop novel computational cameras and sensing techniques that confer extreme vision capabilities to the systems employing them. My work aims to push dynamic 2D and 3D imaging to its limits along multiple axes---3D accuracy, low-light performance, speed, dynamic range, robustness, non-line-of-sight imaging ability, etc---by exploiting two emerging families of computational cameras:
The first---time-resolved SPADs---have pixels sensitive enough to detect individual photons; they are used widely for depth imaging, biological imaging, and looking around corners, and their pixels output a never-ending stream of highly-precise timestamps for the photons they detect. This stream is usually discarded as too massive to process but our initial investigations suggest that the information contained in the timestamp streams makes these cameras extemely powerful: able to measure distances down to microns when combined with a laser, and able to passively record video at speeds almost a billion faster than conventional video cameras.
The second---cameras with pixelwise-programmable exposure---was pioneered by an interdisciplinary team that I built at the University of Toronto, with a vision to provide a "swiss-army-knife sensor" for high-performance 2D and 3D imaging. Now, after more than seven years of pushing the state of the art in speed, pixel size and resolution with new CMOS sensor designs and new camera prototypes, we finally have programmable VGA-resolution sensors with one-of-a-kind capabilities for high-speed, robust 2D and 3D vision.
We seek to unlock the potential of these cameras by (1) laying the foundations for 2D and 3D computer vision at the photon limit, (2) using the flexibility offered by our programmable cameras to solve hard visual inference tasks and (3) integrating sensing, optics and computation even further, to expand our imaging systems' capabilities.