Grant: Discovery Grants Program - Individual
École Polytechnique de Montréal
$70,000.00
Data extraction from videos for the analysis of behaviors in urban scenes
FY 2024-25
Summary
Understanding the behaviors and activities of people, animals and vehicles in urban scenes is very useful to better plan and understand how to build roads, bike paths and public places, such as parks. A good understanding of behaviors can make planning of urban infrastructures more secure, eco-friendly and efficient for its users. To make better decisions in urban planning, a large quantity of data is required. Given that large amount of data, urban planners need tools that can extract useful data automatically. This is the big data problem that we are addressing in this proposal. We want to extract useful data from videos by detecting automatically objects of interest (OOI) and tracking their motion in the scene.
Existing detection methods all fail in various situations. For example, methods based on background subtraction are limited due to the fact that they rely on a color contrast between the OOI and the learnt scene distribution, and they assume that the camera is still. Methods based on optical flow require the OOIs to be moving. Finally, machine learning-based multiclass detectors are limited in detecting only predefined object classes and cannot detect unexpected OOIs. For all those methods, domain adaptation is an issue as they often underperform on new scenarios. Therefore, we hypothesize that these methods should be somehow combined together and care should be taken so that they can be easily adapted to new scenarios. Furthermore, tracking methods are very sensitive to the quality of the OOI detection. Therefore, we want to study jointly OOI detection and tracking to improve both tasks.
The objectives of this proposal are to investigate and propose original approaches to improve OOI detection methods, to combine OOIs detected from several OOIs methods, to find ways to adapt detection methods conveniently to new scenarios, and to design tracking methods that can capitalize better on the information provided by the OOI methods and that can better handle detection errors by performing jointly detection and tracking.
The originality of this research proposal is that we investigate the fusion of object detection methods that are usually studied separately and that we study jointly detection and tracking to improve both tasks. The tools that we will develop in this proposal can be used for any visual surveillance applications.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-04633
Name nom
Bilodeau, Guillaume-Alexandre
Department département
Génie informatique
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
70000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Data extraction from videos for the analysis of behaviors in urban scenes
Keywords
computer vision; machine learning; object detection; visual object tracking; self-supervised learning; video analysis
Application summary
Understanding the behaviors and activities of people, animals and vehicles in urban scenes is very useful to better plan and understand how to build roads, bike paths and public places, such as parks. A good understanding of behaviors can make planning of urban infrastructures more secure, eco-friendly and efficient for its users. To make better decisions in urban planning, a large quantity of data is required. Given that large amount of data, urban planners need tools that can extract useful data automatically. This is the big data problem that we are addressing in this proposal. We want to extract useful data from videos by detecting automatically objects of interest (OOI) and tracking their motion in the scene.
Existing detection methods all fail in various situations. For example, methods based on background subtraction are limited due to the fact that they rely on a color contrast between the OOI and the learnt scene distribution, and they assume that the camera is still. Methods based on optical flow require the OOIs to be moving. Finally, machine learning-based multiclass detectors are limited in detecting only predefined object classes and cannot detect unexpected OOIs. For all those methods, domain adaptation is an issue as they often underperform on new scenarios. Therefore, we hypothesize that these methods should be somehow combined together and care should be taken so that they can be easily adapted to new scenarios. Furthermore, tracking methods are very sensitive to the quality of the OOI detection. Therefore, we want to study jointly OOI detection and tracking to improve both tasks.
The objectives of this proposal are to investigate and propose original approaches to improve OOI detection methods, to combine OOIs detected from several OOIs methods, to find ways to adapt detection methods conveniently to new scenarios, and to design tracking methods that can capitalize better on the information provided by the OOI methods and that can better handle detection errors by performing jointly detection and tracking.
The originality of this research proposal is that we investigate the fusion of object detection methods that are usually studied separately and that we study jointly detection and tracking to improve both tasks. The tools that we will develop in this proposal can be used for any visual surveillance applications.