Grant: Discovery Grants Program - Individual
McMaster University
$25,000.00
Multi-Modality-Enriched Video: Potential, Strategies and Applications
EF 2019-20
Résumé
Visual sensors are ubiquitous and have found their ways into many different applications. They capturing data at an incredible pace and are being integrated with other sensors into single devices. For example a typical smartphone has a camera (visual sensor), a GPS sensor, an accelerometer, a gyroscope, and a magnetometer. These consolidated sensors generate multi-modality-enriched video signal: video signal as well as one or multiple 1-D or 2-D non-video signals. The non-video signals can provide additional context for video signal processing or directly provide side information to facilitate video signal processing and communication. For example a motion sensor can provide motion data useful for motion estimation and thus video compression. Moreover, multi-modality enriched video enables us to develop new functionalities and create new user experiences. The enriching non-video signals have a tremendous potential for improving the way we communicate and make decision based on video. However, realization of this potential hinges upon advancement in techniques for managing, analyzing, and extracting useful information from these multi-modality data sets. Our utilization of interdependency between different signal modality acquired by enriched visual sensors is rather intuitive. Currently, there is no established rationale for determining if there are any benefits to use more than one signal modality in a particular video processing task (e.g. video compression, detection, tracking) and which one to use and how to use it. In this Discovery Grant research, a new line of inquiry will be pursued in order to systematically utilize the inter dependencies between different signal modalities for video compression, post processing, detection and classification. A tensor based framework will be adopted that will characterize the interaction between sensing modalities. The proposed research is expected to yield new approaches for systematic analysis and processing of multi-modality enriched video. The research will lead to superior video communication and better video-based tracking, recognition and classifications. The new knowledge arising from this research will find its way into new technologies and products in areas of great importance to Canada such as education, healthcare, manufacturing and auto industry. Canadian companies in this area will benefit from the new knowledge and also the highly qualified personnel that will participate in the research.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2015-06637
Name nom
Shirani, Shahram
Department département
Electrical and Computer Engineering
Organization id
27
Institution établissement
McMaster University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2015
Award amount
25000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2514
Research subject group en
Electrical and electronic engineering
Research subject group fr
Génie électrique et électronique
Research subject en
Digital signal processing
Research subject fr
Traitement des signaux numériques
Installment
inst:5 - 5
Partie
5
Nb partie
5
Application title
Multi-Modality-Enriched Video: Potential, Strategies and Applications
Keyword
NSERC
Application summary
Visual sensors are ubiquitous and have found their ways into many different applications. They capturing data at an incredible pace and are being integrated with other sensors into single devices. For example a typical smartphone has a camera (visual sensor), a GPS sensor, an accelerometer, a gyroscope, and a magnetometer. These consolidated sensors generate multi-modality-enriched video signal: video signal as well as one or multiple 1-D or 2-D non-video signals. The non-video signals can provide additional context for video signal processing or directly provide side information to facilitate video signal processing and communication. For example a motion sensor can provide motion data useful for motion estimation and thus video compression. Moreover, multi-modality enriched video enables us to develop new functionalities and create new user experiences. The enriching non-video signals have a tremendous potential for improving the way we communicate and make decision based on video. However, realization of this potential hinges upon advancement in techniques for managing, analyzing, and extracting useful information from these multi-modality data sets. Our utilization of interdependency between different signal modality acquired by enriched visual sensors is rather intuitive. Currently, there is no established rationale for determining if there are any benefits to use more than one signal modality in a particular video processing task (e.g. video compression, detection, tracking) and which one to use and how to use it. In this Discovery Grant research, a new line of inquiry will be pursued in order to systematically utilize the inter dependencies between different signal modalities for video compression, post processing, detection and classification. A tensor based framework will be adopted that will characterize the interaction between sensing modalities. The proposed research is expected to yield new approaches for systematic analysis and processing of multi-modality enriched video. The research will lead to superior video communication and better video-based tracking, recognition and classifications. The new knowledge arising from this research will find its way into new technologies and products in areas of great importance to Canada such as education, healthcare, manufacturing and auto industry. Canadian companies in this area will benefit from the new knowledge and also the highly qualified personnel that will participate in the research.