Grant: Discovery Grants Program - Individual
University of Alberta
$46,000.00
Analysing visual behaviours of articulated objects from images and videos
EF 2024-25
Résumé
Every day, an enormous amount of images and video clips are produced by cameras around the world, including our mobile phone cameras. The focus of attention in the images and videos is often about human individuals and their behaviours. As one might expect, the ability to interpret human visual behaviours has become a central theme in computer vision, which has indeed a wide spectrum of applications. At its root is a set of fundamental computer vision problems, including pose estimation and action detection of individuals from images and videos, among others.
Thanks to the recent advances in camera hardware and machine learning technologies, now there are several dedicated systems with impressive results in interpreting human full-body behaviours, and we start to witness their influence in everyday life, ranging from Microsoft Kinect in gaming to the cashier-less Amazon Go grocery stores. My research group has been among the first in developing algorithms and demonstration systems in analysing 3D human hand poses from depth images (2013), as well as 3D lab animal behaviours from a depth camera, such as lab mouse in the open-field arena and zebrafish in the fish tank (2017). Meanwhile, there are still many hurdles to overcome.
The proposed research program is to advance the state-of-the-art in visual behaviour analysis, with the long term goal of enabling machines to properly see and interact with a human inhabited environment. In particular, it aims to address some of the main shortcomings by investigating the following two directions. In the first direction, we focus on devising learning algorithms that follow the geometric constraints of articulated objects, as well as curating comprehensive benchmarks with rich annotations to facilitate the development of holistic approaches. Moreover, instead of elaborating around the traditional frame-based cameras that have the inherit drawbacks including latency, memory, and computational burdens, we plan to examine the synergism between our geometry and learning based approach and the very recent dynamic vision cameras. In the second direction, we investigate important applications that may go beyond human full-bodies. It is of strong interest to look at the behaviours of human hands, as well as animals. It is also timely to review sign language applications that would help disadvantaged people and have significant societal impacts.
Potential impact of this proposal includes (1) addressing the shortcomings of (self-)occlusion, and inconsistent & incomplete annotations, (2) introducing new benchmark for visual behaviour analysis, and (3) expediting the dissemination of new advances in related practical applications, such as visual animal behaviour analysis and visual sign language translation. The HQP trained through this research program will gain research as well as hands-on experiences in computer vision and machine learning that could be valuable in their future academic or industry careers.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-04575
Name nom
Cheng, Li
Department département
Department of Electrical and Computer Engineering
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2019
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Analysing visual behaviours of articulated objects from images and videos
Keywords
pose estimation; action recognition and detection; visual behaviour analysis; image and video analysis; computer vision; deep learning; machine learning
Application summary
Every day, an enormous amount of images and video clips are produced by cameras around the world, including our mobile phone cameras. The focus of attention in the images and videos is often about human individuals and their behaviours. As one might expect, the ability to interpret human visual behaviours has become a central theme in computer vision, which has indeed a wide spectrum of applications. At its root is a set of fundamental computer vision problems, including pose estimation and action detection of individuals from images and videos, among others.
Thanks to the recent advances in camera hardware and machine learning technologies, now there are several dedicated systems with impressive results in interpreting human full-body behaviours, and we start to witness their influence in everyday life, ranging from Microsoft Kinect in gaming to the cashier-less Amazon Go grocery stores. My research group has been among the first in developing algorithms and demonstration systems in analysing 3D human hand poses from depth images (2013), as well as 3D lab animal behaviours from a depth camera, such as lab mouse in the open-field arena and zebrafish in the fish tank (2017). Meanwhile, there are still many hurdles to overcome.
The proposed research program is to advance the state-of-the-art in visual behaviour analysis, with the long term goal of enabling machines to properly see and interact with a human inhabited environment. In particular, it aims to address some of the main shortcomings by investigating the following two directions. In the first direction, we focus on devising learning algorithms that follow the geometric constraints of articulated objects, as well as curating comprehensive benchmarks with rich annotations to facilitate the development of holistic approaches. Moreover, instead of elaborating around the traditional frame-based cameras that have the inherit drawbacks including latency, memory, and computational burdens, we plan to examine the synergism between our geometry and learning based approach and the very recent dynamic vision cameras. In the second direction, we investigate important applications that may go beyond human full-bodies. It is of strong interest to look at the behaviours of human hands, as well as animals. It is also timely to review sign language applications that would help disadvantaged people and have significant societal impacts.
Potential impact of this proposal includes (1) addressing the shortcomings of (self-)occlusion, and inconsistent & incomplete annotations, (2) introducing new benchmark for visual behaviour analysis, and (3) expediting the dissemination of new advances in related practical applications, such as visual animal behaviour analysis and visual sign language translation. The HQP trained through this research program will gain research as well as hands-on experiences in computer vision and machine learning that could be valuable in their future academic or industry careers.