Grant: Discovery Grants Program - Individual
Simon Fraser University
$26,000.00
Physics-based colour models in computer vision, colour science, graphics and multimedia
FY 2020-21
Summary
This work is aimed at physics-based concepts in imaging. The project
components described are specific, but form aspects of long-standing and
deep problems throughout computer vision. The context for this research
is the "fusion" of images. For example, suppose we need a greyscale
("black and white") xerox copy of a colour image: here we are aiming to
fuse the three RGB (red, geen, blue) colour channels into a single grey output. This is a
deceptively difficult problem, and hundreds of academic papers have been
published on the subject. Many other problem domains are similar: e.g.,
fusing RGB plus Near-Infrared (NIR) into a new colour image with better night-vision
properties for driving in the dark. It is understood in the camera-research
community that including a NIR sensor in a leading brand of smartphones is about
to take place.
It turns out that the idea of "contrast" is critical for image fusion -- we need to capture
local changes in the set of images to best combine them. Just what contrast
consists of has been long-debated. In my Spectral Edge (SpE) application (and
spin-off company) a definition of contrast is taken from solid mathematical
foundations. It turns out the the SpE algorithm is applicable in a variety of
situations: for example in medical imaging it is common to develop "image" data
with more than just 3 dimensions like RGB colour. And in satellite imaging it is
common to take "spectral" images with hundreds of values at each pixel location.
The SpE approach disentangles how colour, on the one hand, and change in spatial
position on the other, combine to form contrast.
These calculations all take place in the "gradient" domain -- gradient is the
derivative of the image, in both x and y directions, where derivative in imaging
simply means the change from a pixel to its neighbouring pixel. Now, it is a
classical problem in applied mathematics to transform back to a colour image
the gradient (in each of
R, G, B) that we arrived at by combining contrast. The classic solution for
"reintegrating" the gradient back into image dates to the year 1800. However
that method, even though since elaborated in a number of mathematical approaches,
produces images that display unpleasant artifacts
in the output image. Instead, in this proposal I will look deeper into making use
of the SpE approach, along with a new, computer-science approach to
reintegration into an output image. To date, the SpE method (and patent)
uses mathematics that applies to a whole image: here I propose pixel-oriented
methods that should produce substantially more detail.
The methods developed will have far-reaching impact in that gradient-based
approaches are now the standard approach to problems in computational
photography. As one example, this research will have a major impact on the
problem of bringing daytime image information into nighttime surveillance
images.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2017-03886
Name nom
Drew, Mark
Department département
School of Computing Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2020
Competition year année de concours
2017
Award amount
26000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2708
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Image processing
Research subject fr
Traitement des images
Installment
inst:4-5
Partie
2
Nb partie
5
Application title
Physics-based colour models in computer vision, colour science, graphics and multimedia
Keyword
NSERC
Application summary
This work is aimed at physics-based concepts in imaging. The project
components described are specific, but form aspects of long-standing and
deep problems throughout computer vision. The context for this research
is the "fusion" of images. For example, suppose we need a greyscale
("black and white") xerox copy of a colour image: here we are aiming to
fuse the three RGB (red, geen, blue) colour channels into a single grey output. This is a
deceptively difficult problem, and hundreds of academic papers have been
published on the subject. Many other problem domains are similar: e.g.,
fusing RGB plus Near-Infrared (NIR) into a new colour image with better night-vision
properties for driving in the dark. It is understood in the camera-research
community that including a NIR sensor in a leading brand of smartphones is about
to take place.
It turns out that the idea of "contrast" is critical for image fusion -- we need to capture
local changes in the set of images to best combine them. Just what contrast
consists of has been long-debated. In my Spectral Edge (SpE) application (and
spin-off company) a definition of contrast is taken from solid mathematical
foundations. It turns out the the SpE algorithm is applicable in a variety of
situations: for example in medical imaging it is common to develop "image" data
with more than just 3 dimensions like RGB colour. And in satellite imaging it is
common to take "spectral" images with hundreds of values at each pixel location.
The SpE approach disentangles how colour, on the one hand, and change in spatial
position on the other, combine to form contrast.
These calculations all take place in the "gradient" domain -- gradient is the
derivative of the image, in both x and y directions, where derivative in imaging
simply means the change from a pixel to its neighbouring pixel. Now, it is a
classical problem in applied mathematics to transform back to a colour image
the gradient (in each of
R, G, B) that we arrived at by combining contrast. The classic solution for
"reintegrating" the gradient back into image dates to the year 1800. However
that method, even though since elaborated in a number of mathematical approaches,
produces images that display unpleasant artifacts
in the output image. Instead, in this proposal I will look deeper into making use
of the SpE approach, along with a new, computer-science approach to
reintegration into an output image. To date, the SpE method (and patent)
uses mathematics that applies to a whole image: here I propose pixel-oriented
methods that should produce substantially more detail.
The methods developed will have far-reaching impact in that gradient-based
approaches are now the standard approach to problems in computational
photography. As one example, this research will have a major impact on the
problem of bringing daytime image information into nighttime surveillance
images.