Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Université de Sherbrooke

$48,000.00

Robot Learning and Discovering Through Memorizing Visual and Auditory Interactions

EF 2022-23

Résumé
Since its foundation, Artificial Intelligence (AI) aims at becoming a system science targeting working in the wild, messy world, addressing key challenges such as architecture, learning and evaluation. To accomplish this, human-robot interaction (HRI) brings rich opportunities by addressing the integration challenges of embodied AI. The long term objective of my Discovery Grant (DG) research program is the study of how to integrate all the required decision-making processes, ranging from navigation to task accomplishment and interaction with others (robots or humans), so that robots can operate in real life settings over long periods of time. My strategy consists of designing robots with an increasing set of advanced perceptual, reasoning and action capabilities under real-time execution, robustness, adaptability and scalability, capable of being used in real life settings. In the short term, my research program focuses on learning from the use of spatial-temporal visual and auditory patterns to make the robots derive knowledge from its operation environments. Memory plays a central role in recognizing patterns and predicting upcoming percepts and action consequences, which is a key feature and critical component of intelligence. Building on my work on vision-based navigation and episodic memory models, along with cross-referenced information from visual and audio data (using spatial auditory data linked to visual data), I will study how to increase the ability of an autonomous robot to derive concepts, predict future events and elaborate behavioral strategies, by operating in dynamic conditions and interacting with people. Deep Neural Networks (DNNs) reveal to be powerful tools to learn the complexities of visual and audio data, which can be beneficial in providing cues and indications of what could be interesting elements in what the robot is experiencing. Using DNNs designed to detect objects, people, faces, pose, sounds and voice, I will use different representations/models to memorize the interaction history from which to derive knowledge and understanding. Experimentation involves conducting trials in long-lasting HRI scenarios using commercially available and custom-designed robots. Involving 2 postdoc, 2 PhDs, 2 Master's and 2 undergraduates, my research program integrates a large set of components, from vision and audio processing to robot control architecture, memory models, decision-making processes and HRI, with contributions to the fields of visuo-auditory cognition and semantic interpretation of embodied multimodal interaction. Its impacts range from applications on factory assembly lines to healthcare, rehabilitation and aging, and surveillance. All my research contributions funded by the DG program are open source, allowing sharing and contributing to the joint effort of bringing robots closer to people, improving quality of life and our understanding and ability of designing truly intelligent robots.
Programme
Discovery Grants Program - Individual
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2022-04036
Name nom
Michaud, François
Department département
Génie électrique et génie informatique - Génie électrique et génie informatique
Organization id
68
Institution établissement
Université de Sherbrooke
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2022
Award amount
48000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2600
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Robotics
Research subject fr
Robotique
Installment
inst:1
Partie
1
Nb partie
5
Application title
Robot Learning and Discovering Through Memorizing Visual and Auditory Interactions
Application summary
Since its foundation, Artificial Intelligence (AI) aims at becoming a system science targeting working in the wild, messy world, addressing key challenges such as architecture, learning and evaluation. To accomplish this, human-robot interaction (HRI) brings rich opportunities by addressing the integration challenges of embodied AI. The long term objective of my Discovery Grant (DG) research program is the study of how to integrate all the required decision-making processes, ranging from navigation to task accomplishment and interaction with others (robots or humans), so that robots can operate in real life settings over long periods of time. My strategy consists of designing robots with an increasing set of advanced perceptual, reasoning and action capabilities under real-time execution, robustness, adaptability and scalability, capable of being used in real life settings. In the short term, my research program focuses on learning from the use of spatial-temporal visual and auditory patterns to make the robots derive knowledge from its operation environments. Memory plays a central role in recognizing patterns and predicting upcoming percepts and action consequences, which is a key feature and critical component of intelligence. Building on my work on vision-based navigation and episodic memory models, along with cross-referenced information from visual and audio data (using spatial auditory data linked to visual data), I will study how to increase the ability of an autonomous robot to derive concepts, predict future events and elaborate behavioral strategies, by operating in dynamic conditions and interacting with people. Deep Neural Networks (DNNs) reveal to be powerful tools to learn the complexities of visual and audio data, which can be beneficial in providing cues and indications of what could be interesting elements in what the robot is experiencing. Using DNNs designed to detect objects, people, faces, pose, sounds and voice, I will use different representations/models to memorize the interaction history from which to derive knowledge and understanding. Experimentation involves conducting trials in long-lasting HRI scenarios using commercially available and custom-designed robots. Involving 2 postdoc, 2 PhDs, 2 Master's and 2 undergraduates, my research program integrates a large set of components, from vision and audio processing to robot control architecture, memory models, decision-making processes and HRI, with contributions to the fields of visuo-auditory cognition and semantic interpretation of embodied multimodal interaction. Its impacts range from applications on factory assembly lines to healthcare, rehabilitation and aging, and surveillance. All my research contributions funded by the DG program are open source, allowing sharing and contributing to the joint effort of bringing robots closer to people, improving quality of life and our understanding and ability of designing truly intelligent robots.