Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Idea to Innovation

University of Toronto

$125,000.00

Motion-tolerant coded-exposure-pixel cameras for emerging computational photography applications (Phase I)

FY 2021-22

Summary
Motion causes significant imaging artifacts, which become prohibitively large when complex computational photography algorithms are implemented on conventional cameras, as they often require the use of multiple video frames. Highly-specialized high-frame-rate image sensors can be used to perform CEP-like imaging without such artifacts. But such high-speed cameras are prone to high noise, excessive power consumption, and extremely high output data rate, and thus are typically bulky and very expensive. We are introducing an emerging class of coded-exposure pixel (CEP) image sensors, that keep motion artifacts low and eliminate other existing drawbacks by implementing exposure-time programmability for each individual pixel.Technically speaking, for each (relatively slow) frame readout many (fast) exposures are performed, with the photogenerated charge for each exposure directed to one of several accumulation nodes within the pixel, based on a pixel code. This simple in-pixel charge-sorting function comes at a cost of an inconsequential pixel area overhead (especially for stacked-wafer image sensors), but allows for implementing a very powerful set of computational photography applications, without the drawbacks of the other approaches. The objective of this project is to refine and commercialize our CEP image sensors - microchips that are at the heart of a new class of motion-tolerant computational cameras. CEP image sensors dynamically program the exposure time of each individual pixel as opposed to conventional cameras where all pixels are exposed to light over equal timeintervals.Specifically, we target three imaging capabilities: (1) adaptive-pixel-exposure imaging under wide-ranging illumination conditions, (2) dynamic 3D imaging, and (3) sub-surface imaging of translucent objects, all done for moving scenes but without exacerbating motion artifacts. These and many other transformative imaging technologies are studied within the field of computational photography, an area of computer vision where synergetic use of imaging and computation yields powerful vision capabilities.
Program
Idea to Innovation
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
561521-2021
Name nom
Genov, Roman
Department département
Electrical & Computer Engineering, Edward S. Rogers Sr. Department of
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2021
Award amount
125000
Program id
I2IPJ
Program name en
Idea to Innovation
Program name fr
De l'idée à l'innovation
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
513
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Electrical and electronic machinery and equipment (including computer hardware)
Area of application fr
Machinerie et équipement électriques et électroniques (y compris matériel informatique)
Research subject code
2504
Research subject group en
Electrical and electronic engineering
Research subject group fr
Génie électrique et électronique
Research subject en
Integrated circuits
Research subject fr
Circuits intégrés
Installment
inst:1
Partie
1
Nb partie
1
Application title
Motion-tolerant coded-exposure-pixel cameras for emerging computational photography applications (Phase I)
Keyword
NSERC
Application summary
Motion causes significant imaging artifacts, which become prohibitively large when complex computational photography algorithms are implemented on conventional cameras, as they often require the use of multiple video frames. Highly-specialized high-frame-rate image sensors can be used to perform CEP-like imaging without such artifacts. But such high-speed cameras are prone to high noise, excessive power consumption, and extremely high output data rate, and thus are typically bulky and very expensive. We are introducing an emerging class of coded-exposure pixel (CEP) image sensors, that keep motion artifacts low and eliminate other existing drawbacks by implementing exposure-time programmability for each individual pixel.Technically speaking, for each (relatively slow) frame readout many (fast) exposures are performed, with the photogenerated charge for each exposure directed to one of several accumulation nodes within the pixel, based on a pixel code. This simple in-pixel charge-sorting function comes at a cost of an inconsequential pixel area overhead (especially for stacked-wafer image sensors), but allows for implementing a very powerful set of computational photography applications, without the drawbacks of the other approaches. The objective of this project is to refine and commercialize our CEP image sensors - microchips that are at the heart of a new class of motion-tolerant computational cameras. CEP image sensors dynamically program the exposure time of each individual pixel as opposed to conventional cameras where all pixels are exposed to light over equal timeintervals.Specifically, we target three imaging capabilities: (1) adaptive-pixel-exposure imaging under wide-ranging illumination conditions, (2) dynamic 3D imaging, and (3) sub-surface imaging of translucent objects, all done for moving scenes but without exacerbating motion artifacts. These and many other transformative imaging technologies are studied within the field of computational photography, an area of computer vision where synergetic use of imaging and computation yields powerful vision capabilities.