Grant: Discovery Grants Program - Individual
Carleton University
$28,000.00
Towards Accurate Detection, Segmentation, and Tracking of Objects and other Scene Attributes
FY 2023-24
Summary
There is benefit in being able to identify, segment, and track individual objects in video, including rigid and articulated objects (e.g. humans, animals, cars, etc.), background elements (sky, clouds, trees, etc.), markers (i.e. objects specifically design to show as high-contrast to a camera), and feature points (fixed, high-contrast pixels, usually corners, that can be easily tracked over multiple frames and used as reference points). Apart from being able to accurately assign each pixel to a specific object, it is useful to determine where they are placed (in true 3D space if possible), and keep track of them through multiple video frames even as their appearance changes or they are partially/fully occluded by other objects as they move through the scene. This technology is used in a range of applications, including: video compression, depth estimation from single and stereo cameras (e.g. robotics), multi-camera stitching (to compensate for parallax), match-moving for visual effects, automatic camera calibration systems, marker-based and marker-less motion capture, and autonomous vehicle vision systems to name a few. The limitation of current systems is that it is difficult to extract the exact outline that distinguishes between two objects (especially as one pixel can represent a blending of both). The reasons for this include: lack of contrast, colour variation, texture, or lighting; frequent shifts in camera perspective and object orientation; motion blur (due to a fast moving scene and/or camera, or low light - meaning slower shutter speeds); light, shadow, and reflection variations (both under static and changing conditions), and various other smaller artifacts that are inherent within the camera (including sensor noise and simply the fact most sensors are Bayer-tile type and thus frames are constructed through demosaicing). In addition, most applications require real-time processing so that other systems can react to the information (e.g. autonomous vehicles, perimeter security). Therefore, identifying individual objects, determining their shape and outlines to sub-pixel accuracy and accurately tracking them in a sequence of frames is crucial for many mission-critical applications, especially under varying adverse conditions (e.g. snow or rain), without the need to rely on additional costly technology (such as LIDAR/RADAR) In this research program, we propose a range of new and complementary research directions based on a combination of traditional video processing techniques and deep learning methods that can be used to improve the accuracy of the current state of the art and deal with the more varied and complex conditions that can be expected in the application areas especially where the conditions cannot be controlled, but where they can be relied upon and trusted to produce precise real-time results.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2021-04248
Name nom
Joslin, Chris
Department département
Information Technology, School of - Information Technology, School of
Organization id
23
Institution établissement
Carleton University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2705
Research subject group en
Information technology
Research subject group fr
Software and development
Research subject en
Technologie de l'information
Research subject fr
Logiciels et développement
Application title
Towards Accurate Detection, Segmentation, and Tracking of Objects and other Scene Attributes
Keyword
Light, Shadow, and Shade Detection; Multi-Camera Systems; Object Segmentation; Object Tracking
Application summary
There is benefit in being able to identify, segment, and track individual objects in video, including rigid and articulated objects (e.g. humans, animals, cars, etc.), background elements (sky, clouds, trees, etc.), markers (i.e. objects specifically design to show as high-contrast to a camera), and feature points (fixed, high-contrast pixels, usually corners, that can be easily tracked over multiple frames and used as reference points). Apart from being able to accurately assign each pixel to a specific object, it is useful to determine where they are placed (in true 3D space if possible), and keep track of them through multiple video frames even as their appearance changes or they are partially/fully occluded by other objects as they move through the scene. This technology is used in a range of applications, including: video compression, depth estimation from single and stereo cameras (e.g. robotics), multi-camera stitching (to compensate for parallax), match-moving for visual effects, automatic camera calibration systems, marker-based and marker-less motion capture, and autonomous vehicle vision systems to name a few. The limitation of current systems is that it is difficult to extract the exact outline that distinguishes between two objects (especially as one pixel can represent a blending of both). The reasons for this include: lack of contrast, colour variation, texture, or lighting; frequent shifts in camera perspective and object orientation; motion blur (due to a fast moving scene and/or camera, or low light - meaning slower shutter speeds); light, shadow, and reflection variations (both under static and changing conditions), and various other smaller artifacts that are inherent within the camera (including sensor noise and simply the fact most sensors are Bayer-tile type and thus frames are constructed through demosaicing). In addition, most applications require real-time processing so that other systems can react to the information (e.g. autonomous vehicles, perimeter security). Therefore, identifying individual objects, determining their shape and outlines to sub-pixel accuracy and accurately tracking them in a sequence of frames is crucial for many mission-critical applications, especially under varying adverse conditions (e.g. snow or rain), without the need to rely on additional costly technology (such as LIDAR/RADAR) In this research program, we propose a range of new and complementary research directions based on a combination of traditional video processing techniques and deep learning methods that can be used to improve the accuracy of the current state of the art and deal with the more varied and complex conditions that can be expected in the application areas especially where the conditions cannot be controlled, but where they can be relied upon and trusted to produce precise real-time results.