Grant: Discovery Grants Program - Individual
University of Toronto
$59,000.00
Fast Field-Programmable Coded Image Sensors for Versatile Low-Cost Computational Imaging
EF 2023-24
Résumé
Today's smartphone cameras are very good and inexpensive. This is mainly because they use computational imaging techniques to digitally enhance images by means of software processing. A common computational imaging technique is to take multiple shots and combine them into one enhanced image. However, this approach fails in applications where fast motion or rapidly changing illumination are present, such as drones, automated vehicles, surveillance, robotics and augmented reality. So, future image sensors will have to rely on a combination of fast imaging and computation to tolerate motion and maintain the same low cost as in smartphones. There are two main classes of cameras: the first class is the conventional consumer cameras such as cameras in most smartphones. They are inexpensive, well below $50, but have relatively slow exposure, usually below 300 exposures per second, and therefore, do not tolerate fast motion well. This leads to computational imaging artifacts such as ghosting when, for example, instead of one flying bird the sensor captures as many displaced images of the bird as the number of shots in a burst. These cameras are also inflexible, usually designed to implement only one active computational imaging technique. On the other end of the spectrum are the high-speed cameras. These cameras can operate over 100 times faster to avoid motion artifacts, but at a proportionally higher cost, at $5,000 or more. The cost is dominated by the hardware needed to handle and process the large and high-speed video output data. In addition, these cameras require too much electric power and extremely bright illumination. All these limitations arise from a single key property of both of these camera types - only one exposure is performed for one readout. The proposed solution is a new class of image sensors that are motion tolerant, low-cost, and versatile, and as such are well positioned to support robust computational imaging when the incident light intensity changes rapidly. This is enabled by their high exposure rate, with over 12.5M exposures per second, which is a factor of 40,000x higher than the best cell phone camera. The readout speed is maintained beneficially low - at the standard video rate. As there is no high-speed video output, no expensive hardware is needed to handle it, so the cost can be well below $50 or less than 1% of the cost of conventional high-speed cameras. The slow readout also brings the benefits of low power dissipation and much lower required illumination. Additionally, our coded sensors are highly versatile in their functionality and thus are application-agnostic. They are reconfigurable by the end-user in the field to implement one of many imaging computational techniques, simply by changing pixel codes in firmware. So, the same sensors can serve multiple different applications and markets, serving as an "all-in-one solution" for various needs of computational imaging.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2023-05797
Name nom
Genov, Roman
Department département
Electrical Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2023
Award amount
59000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
513
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Electrical and electronic machinery and equipment (including computer hardware)
Area of application fr
Machinerie et équipement électriques et électroniques (y compris matériel informatique)
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Computer vision
Research subject en
Robotique
Research subject fr
Vision artificielle
Application title
Fast Field-Programmable Coded Image Sensors for Versatile Low-Cost Computational Imaging
Keyword
Analog integrated curcuits; Cameras; Computational imaging; Computer vision; High-dynamic-range imaging; High-speed imaging; Imager; Image sensors; Integrated circuits; Mixed-signal integrated circuits
Application summary
Today's smartphone cameras are very good and inexpensive. This is mainly because they use computational imaging techniques to digitally enhance images by means of software processing. A common computational imaging technique is to take multiple shots and combine them into one enhanced image. However, this approach fails in applications where fast motion or rapidly changing illumination are present, such as drones, automated vehicles, surveillance, robotics and augmented reality. So, future image sensors will have to rely on a combination of fast imaging and computation to tolerate motion and maintain the same low cost as in smartphones. There are two main classes of cameras: the first class is the conventional consumer cameras such as cameras in most smartphones. They are inexpensive, well below $50, but have relatively slow exposure, usually below 300 exposures per second, and therefore, do not tolerate fast motion well. This leads to computational imaging artifacts such as ghosting when, for example, instead of one flying bird the sensor captures as many displaced images of the bird as the number of shots in a burst. These cameras are also inflexible, usually designed to implement only one active computational imaging technique. On the other end of the spectrum are the high-speed cameras. These cameras can operate over 100 times faster to avoid motion artifacts, but at a proportionally higher cost, at $5,000 or more. The cost is dominated by the hardware needed to handle and process the large and high-speed video output data. In addition, these cameras require too much electric power and extremely bright illumination. All these limitations arise from a single key property of both of these camera types - only one exposure is performed for one readout. The proposed solution is a new class of image sensors that are motion tolerant, low-cost, and versatile, and as such are well positioned to support robust computational imaging when the incident light intensity changes rapidly. This is enabled by their high exposure rate, with over 12.5M exposures per second, which is a factor of 40,000x higher than the best cell phone camera. The readout speed is maintained beneficially low - at the standard video rate. As there is no high-speed video output, no expensive hardware is needed to handle it, so the cost can be well below $50 or less than 1% of the cost of conventional high-speed cameras. The slow readout also brings the benefits of low power dissipation and much lower required illumination. Additionally, our coded sensors are highly versatile in their functionality and thus are application-agnostic. They are reconfigurable by the end-user in the field to implement one of many imaging computational techniques, simply by changing pixel codes in firmware. So, the same sensors can serve multiple different applications and markets, serving as an "all-in-one solution" for various needs of computational imaging.