Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Engage Grants Program

École de technologie supérieure

$25,000.00

Fusion of Face and Speech Characteristics for Real-Time Recognition of Behaviors in Human-Centric E-Learning

EF 2015-16

Résumé
Over the past few decades, several new technologies have been adopted to enhance online e-learning programs. NVS Learning Inc. seeks to deliver interactive training programs based on virtual and immersive environments that develop positive behavioral skills, as needed in effective leadership, management, customer service, sales, etc. Although still underexploited, recent developments webcam and microphone technologies, as well as signal processing and pattern recognition techniques would allow for more natural and unobtrusive interactions with such environments. However, commercial systems that exploit these technologies are limited to offline post-analysis of face and speech information and recognition of a small set of expressions. Real-time recognition of behavioral skills based on faces and speech captured in videos is a challenging large-scale e-learning application. This project investigates state-of-the-art techniques that are suitable for the real-time recognition of behavioral skills in e-learning based on face and speech modalities. Given a video stream that captures the variations in the learner's face and speech during interactions with an e-learning environment, an automated system is required to learn and interpret his/her behavioral skills, e.g., the flow of the learner's speech verbalizations, and provide the learner with personalised feedback. Several advanced signal processing and pattern techniques will be reviewed and compared. In particular, patch-based sparse representation, multiple instance learning, and ensemble learning techniques will be considered to design a pool of behavior detectors. This project aims to provide NVS Learning with the expertise required to deploy robust interactive e-learning systems for customer applications. Benchmarking of promising techniques will be performed with videos from the RECOLA public dataset, and performance will be assessed in terms of accuracy and resource requirements.
Programme
Engage Grants Program
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
485329-2015
Name nom
Granger, Eric
Department département
Génie de la production automatisée
Organization id
53
Institution établissement
École de technologie supérieure
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2015
Competition year année de concours
2015
Award amount
25000
Program id
EGP
Program name en
Engage Grants Program
Program name fr
Programme de subventions d'engagement partenariat
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1552
Committee name en
Quebec Internal Decision Committee
Committee name fr
Comité de décision interne pour le Québec
Area of application code
1103
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Learning and education
Area of application fr
Apprentissage et éducation
Research subject code
2605
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Pattern analysis and machine intelligence
Research subject fr
Analyse de formes et intelligence artificielle
Partie
1
Nb partie
1
Application title
Fusion of Face and Speech Characteristics for Real-Time Recognition of Behaviors in Human-Centric E-Learning
Keyword
NSERC
Application summary
Over the past few decades, several new technologies have been adopted to enhance online e-learning programs. NVS Learning Inc. seeks to deliver interactive training programs based on virtual and immersive environments that develop positive behavioral skills, as needed in effective leadership, management, customer service, sales, etc. Although still underexploited, recent developments webcam and microphone technologies, as well as signal processing and pattern recognition techniques would allow for more natural and unobtrusive interactions with such environments. However, commercial systems that exploit these technologies are limited to offline post-analysis of face and speech information and recognition of a small set of expressions. Real-time recognition of behavioral skills based on faces and speech captured in videos is a challenging large-scale e-learning application. This project investigates state-of-the-art techniques that are suitable for the real-time recognition of behavioral skills in e-learning based on face and speech modalities. Given a video stream that captures the variations in the learner's face and speech during interactions with an e-learning environment, an automated system is required to learn and interpret his/her behavioral skills, e.g., the flow of the learner's speech verbalizations, and provide the learner with personalised feedback. Several advanced signal processing and pattern techniques will be reviewed and compared. In particular, patch-based sparse representation, multiple instance learning, and ensemble learning techniques will be considered to design a pool of behavior detectors. This project aims to provide NVS Learning with the expertise required to deploy robust interactive e-learning systems for customer applications. Benchmarking of promising techniques will be performed with videos from the RECOLA public dataset, and performance will be assessed in terms of accuracy and resource requirements.