Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Engage Grants Program

University of Toronto

$25,000.00

Hand Segmentation and Detection Applied to Mobile Devices Using a Noisy Input Depth Image Stream****

EF 2018-19

Résumé
Human computer interaction is rapidly evolving with an emerging focus on gesture recognition tools. There are several state-of-the-art algorithms that tackle aspects of this problem, but no convincing result has been presented that tracks motion, localizes hands, and interprets gestures. Leap Motion specializes in hand tracking but does not produce suitable gesture detection and interpretation. Mobile devices such as iPhone are equipped with depth cameras, but the industry needs innovative, improved algorithms to produce an accurate 3D representation of a moving hand. Hence, accurate real-time human hand tracking represents a fertile area of advanced research and technology development, with no robust solution to date.* Toronto-based Xesto has developed a cloud-based machine learning platform to recognize and record gestures for use in gesture-based applications using depth map from Leap Motion and mobile devices with depth cameras. Xesto has built a prototype that currently requires a superior 3D representation of the hand, while leveraging robust hand detection algorithms on Xesto's device agnostic platform.* This project represents a collaboration between Professor Deepa Kundur and Xesto to improve the real time performance of hand tracking on mobile devices. Specifically, the project makes use of Xesto's RGB-D hand-pose detection and gesture recognition pipeline that is applicable to any depth sensor. The research aims to design a novel convolutional neural network architecture that uses temporal information to produce an accurate real-time heat map of hand-features. This problem has a high level of technical depth in estimating and optimizing the energy function that constructs articulable objects with large degrees of freedom, self-similar parts and is subject to self-occlusion. * The proposed research is expected to have impact in the field of hand tracking providing Xesto a technological advantage to develop superior applications for health, retail and automotive industry. Further, this work will grant each mobile device user a new way to interact with their machines only through gestures.
Programme
Engage Grants Program
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
537630-2018
Name nom
Kundur, Deepa
Department département
Electrical & Computer Engineering, Edward S. Rogers Sr. Department of
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2018
Competition year année de concours
2018
Award amount
25000
Program id
EGP
Program name en
Engage Grants Program
Program name fr
Programme de subventions d'engagement partenariat
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1553
Committee name en
Ontario Internal Decision Committee
Committee name fr
Comité de décision interne pour l'Ontario
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2514
Research subject group en
Electrical and electronic engineering
Research subject group fr
Génie électrique et électronique
Research subject en
Digital signal processing
Research subject fr
Traitement des signaux numériques
Partie
1
Nb partie
1
Application title
Hand Segmentation and Detection Applied to Mobile Devices Using a Noisy Input Depth Image Stream****
Keyword
NSERC
Application summary
Human computer interaction is rapidly evolving with an emerging focus on gesture recognition tools. There are several state-of-the-art algorithms that tackle aspects of this problem, but no convincing result has been presented that tracks motion, localizes hands, and interprets gestures. Leap Motion specializes in hand tracking but does not produce suitable gesture detection and interpretation. Mobile devices such as iPhone are equipped with depth cameras, but the industry needs innovative, improved algorithms to produce an accurate 3D representation of a moving hand. Hence, accurate real-time human hand tracking represents a fertile area of advanced research and technology development, with no robust solution to date.* Toronto-based Xesto has developed a cloud-based machine learning platform to recognize and record gestures for use in gesture-based applications using depth map from Leap Motion and mobile devices with depth cameras. Xesto has built a prototype that currently requires a superior 3D representation of the hand, while leveraging robust hand detection algorithms on Xesto's device agnostic platform.* This project represents a collaboration between Professor Deepa Kundur and Xesto to improve the real time performance of hand tracking on mobile devices. Specifically, the project makes use of Xesto's RGB-D hand-pose detection and gesture recognition pipeline that is applicable to any depth sensor. The research aims to design a novel convolutional neural network architecture that uses temporal information to produce an accurate real-time heat map of hand-features. This problem has a high level of technical depth in estimating and optimizing the energy function that constructs articulable objects with large degrees of freedom, self-similar parts and is subject to self-occlusion. * The proposed research is expected to have impact in the field of hand tracking providing Xesto a technological advantage to develop superior applications for health, retail and automotive industry. Further, this work will grant each mobile device user a new way to interact with their machines only through gestures.