Grant: Discovery Grants Program - Individual
École Polytechnique de Montréal
$18,000.00
Improving visual surveillance using infrared and visible imagery
FY 2016-17
Summary
Automatically detecting people for monitoring and video surveillance purposes is a very difficult problem to solve. Indeed, people in visible (Vis) imagery can be hard to detect when the color contrast with the background is low and when the illumination of the scene is poor. A possible solution is to add an infrared (IR) camera to monitor a scene. Because an infrared camera detects heat, it can detect successfully people in situations complementary to those of a visible camera.
As a result, it is beneficial to use both camera types jointly to capitalize on their respective strengths. In this research, long wavelength IR and visible cameras are used in a stereo configuration, which can also provide coarse depth information. Both planar and non-planar scenes are considered. As well as helping in people detection, an additional benefit of combining IR/Vis is the ability to detect and segment objects carried by humans, as they often contrast strongly in temperature. Finally, thermal information can be used to detect symptoms like fever.
Combining visible and IR to improve the detection of people requires solving many problems. It is especially difficult to find features that match in both modalities to perform registration. Even after registration, both aligned silhouettes (IR/Vis) will be imperfectly segmented. Which parts correspond to the human, which parts do not? My research objectives match these research problems and aim at designing solutions for combining IR/Vis human silhouettes for efficient detection and tracking of people. My long-term objective will be achieved through realization of my short-term objectives:
1) Develop novel detection and tracking methods to facilitate IR/Vis registration;
2) Develop novel methods to improve the speed and accuracy of IR/Vis registration of people silhouettes;
3) Develop novel methods to combine efficiently aligned IR/Vis silhouettes of people;
To reach these objectives, new methods will be developed in object detection, tracking, registration and segmentation. Object detection is essential to have initial segmentation of people. Then registration is used to align visible and infrared human silhouettes. Tracking allows propagating segmentation and registration results from frame to frame. Finally, a final joint segmentation needs to be performed to combine properly the visible and infrared data.
Research resulting from this program will bring novel ideas to the computer vision community to solve practical real-world problems. My research team is expected to make its developed source codes and datasets publicly available to stimulate wider dissemination to end-users and other researchers. All aspects of this research program will involve students (HQPs) that will be trained to learn skills that are in high demand across Canada.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2015-05350
Name nom
Bilodeau, GuillaumeAlexandre
Department département
Génie informatique et génie logiciel - Génie informatique et génie logiciel
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2016
Competition year année de concours
2015
Award amount
18000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Partie
2
Nb partie
5
Application title
Improving visual surveillance using infrared and visible imagery
Keyword
NSERC
Application summary
Automatically detecting people for monitoring and video surveillance purposes is a very difficult problem to solve. Indeed, people in visible (Vis) imagery can be hard to detect when the color contrast with the background is low and when the illumination of the scene is poor. A possible solution is to add an infrared (IR) camera to monitor a scene. Because an infrared camera detects heat, it can detect successfully people in situations complementary to those of a visible camera.
As a result, it is beneficial to use both camera types jointly to capitalize on their respective strengths. In this research, long wavelength IR and visible cameras are used in a stereo configuration, which can also provide coarse depth information. Both planar and non-planar scenes are considered. As well as helping in people detection, an additional benefit of combining IR/Vis is the ability to detect and segment objects carried by humans, as they often contrast strongly in temperature. Finally, thermal information can be used to detect symptoms like fever.
Combining visible and IR to improve the detection of people requires solving many problems. It is especially difficult to find features that match in both modalities to perform registration. Even after registration, both aligned silhouettes (IR/Vis) will be imperfectly segmented. Which parts correspond to the human, which parts do not? My research objectives match these research problems and aim at designing solutions for combining IR/Vis human silhouettes for efficient detection and tracking of people. My long-term objective will be achieved through realization of my short-term objectives:
1) Develop novel detection and tracking methods to facilitate IR/Vis registration;
2) Develop novel methods to improve the speed and accuracy of IR/Vis registration of people silhouettes;
3) Develop novel methods to combine efficiently aligned IR/Vis silhouettes of people;
To reach these objectives, new methods will be developed in object detection, tracking, registration and segmentation. Object detection is essential to have initial segmentation of people. Then registration is used to align visible and infrared human silhouettes. Tracking allows propagating segmentation and registration results from frame to frame. Finally, a final joint segmentation needs to be performed to combine properly the visible and infrared data.
Research resulting from this program will bring novel ideas to the computer vision community to solve practical real-world problems. My research team is expected to make its developed source codes and datasets publicly available to stimulate wider dissemination to end-users and other researchers. All aspects of this research program will involve students (HQPs) that will be trained to learn skills that are in high demand across Canada.