Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Université de Sherbrooke

$28,000.00

Robot Audition in Everyday Environments with Deep Learning

EF 2024-25

Résumé
Humans and animals rely on audition to monitor their environment, to detect threats and to communicate. In particular, hearing acts as instantaneous omnidirectional attention mechanism, while vision has more continuous and directed field of view. Robots should have similar hearing capabilities to naturally interact with their environment. It is a challenging task to address noisy and dynamic environments, i.e. in the wild and messy world we live in, with the robot having to perform all computations in real-time. Robots audition will allow machines to focus their attention on high priority events not perceptible by vision. Humans could also give robots voice commands when no other interface is available. I believe that large scale deployment of robots in everyday environments will be possible if intelligent machines have advanced hearing capabilities. The goal of this Discovery Grant is to investigate new methods to provide robots with hearing capabilities similar or superior to humans in real world setting. Deep learning showed promising results in audio processing but relies on large datasets and involves high computing power at test time. Deep learning methods need to be adapted for robots as both the embedded computing power and the amount of training data are usually limited. To accomplish this, and based on my current research background, I plan to address the following four short-term objectives: 1) develop a transfer learning method to perform few-shot ego-noise learning; 2) perform sound event localization based on large weakly-labeled datasets; 3) enhance a speech source with neural networks estimating time-frequency masks for each pair of microphones for an array of arbitrary shape; 4) optimize the previous neural networks for real-time processing on an embedded system. The methods will be validated on a wheeled robot that interacts with its environment in realistic settings. To maximize the impact on the robotics community, all the methods will be integrated in an open source software framework. This program will train 4 Highly Qualified Personnel (HQPs) (2 PhDs and 2 MScAs) and 4 undergrad coop trainees. The expertise developed by the HPQs will be based on a unique blend of signal processing, machine learning and embedded system design, which is in high demand amongst Canadian companies involved in the field of robotics. Robots are believed to be part of the solutions to deal with the aging population and labor shortage in developed countries. Robot audition would allow machine to collaborate with workers in warehouses and smart plants and make robotic technologies safer for seniors and healthcare. There is also some serious concern regarding privacy with smart speakers as the technology relies on cloud computing. My research program goes beyond robotics and can benefit this industry as some part of the current research aims to perform speech enhancement on low-cost embedded hardware.
Programme
Discovery Grants Program - Individual
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2021-03908
Name nom
Grondin, François
Department département
Département de Génie Électrique et Génie Informatique
Organization id
68
Institution établissement
Université de Sherbrooke
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2021
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
500
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Manufacturing processes and products
Area of application fr
Procédés et produits industriels
Research subject code
2605
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Pattern analysis and machine intelligence
Research subject fr
Analyse de formes et intelligence artificielle
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Robot Audition in Everyday Environments with Deep Learning
Keywords
robot audition; ego-noise; microphone array; few-shot learning; deep learning; sound classification; speech enhancement; real-time embedded systems; sound localization; transfer learning
Application summary
Humans and animals rely on audition to monitor their environment, to detect threats and to communicate. In particular, hearing acts as instantaneous omnidirectional attention mechanism, while vision has more continuous and directed field of view. Robots should have similar hearing capabilities to naturally interact with their environment. It is a challenging task to address noisy and dynamic environments, i.e. in the wild and messy world we live in, with the robot having to perform all computations in real-time. Robots audition will allow machines to focus their attention on high priority events not perceptible by vision. Humans could also give robots voice commands when no other interface is available. I believe that large scale deployment of robots in everyday environments will be possible if intelligent machines have advanced hearing capabilities. The goal of this Discovery Grant is to investigate new methods to provide robots with hearing capabilities similar or superior to humans in real world setting. Deep learning showed promising results in audio processing but relies on large datasets and involves high computing power at test time. Deep learning methods need to be adapted for robots as both the embedded computing power and the amount of training data are usually limited. To accomplish this, and based on my current research background, I plan to address the following four short-term objectives: 1) develop a transfer learning method to perform few-shot ego-noise learning; 2) perform sound event localization based on large weakly-labeled datasets; 3) enhance a speech source with neural networks estimating time-frequency masks for each pair of microphones for an array of arbitrary shape; 4) optimize the previous neural networks for real-time processing on an embedded system. The methods will be validated on a wheeled robot that interacts with its environment in realistic settings. To maximize the impact on the robotics community, all the methods will be integrated in an open source software framework. This program will train 4 Highly Qualified Personnel (HQPs) (2 PhDs and 2 MScAs) and 4 undergrad coop trainees. The expertise developed by the HPQs will be based on a unique blend of signal processing, machine learning and embedded system design, which is in high demand amongst Canadian companies involved in the field of robotics. Robots are believed to be part of the solutions to deal with the aging population and labor shortage in developed countries. Robot audition would allow machine to collaborate with workers in warehouses and smart plants and make robotic technologies safer for seniors and healthcare. There is also some serious concern regarding privacy with smart speakers as the technology relies on cloud computing. My research program goes beyond robotics and can benefit this industry as some part of the current research aims to perform speech enhancement on low-cost embedded hardware.