Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

Université de Sherbrooke

$46,000.00

Enabling rich, open-ended human-robot interaction through robust, advanced multimodal perceptual capabilities for high-level reasoning

FY 2024-25

Summary
The ultimate goal in Human-Robot Interaction is to have autonomous robots that can naturally interact with people, assist them in their daily lives at home or at work, and automatically adapt to new situations. Building such robots brings a great number of scientific challenges: safe, natural and effective interaction implies advanced perceptual capabilities to supply information for sound and robust reasoning. As humans operate in dynamic environments, robots will need a broad understanding of the world we live in and reasoning capabilities outside of the usually rigid, pre-defined scenarios they are currently programmed for. My research program is oriented toward enabling rich, open-ended interactions between humans and autonomous mobile robots. Doing so require high-level reasoning, which I believe is only possible with large-scale and multimodal perceptual capabilities on robots. To provide such capabilities, and as identified by studies on selective attention (SA) in neurosciences, there is growing evidence that top-down modulation occurs in the brain to maintain performance of the working memory by prioritizing already encoded items depending on the changing tasks demands and expectations of dynamic environments. A robot meant for rich interaction will have to adapt itself to such environments and prioritize what needs to be encoded in its own working memory. Furthermore, it will have to anticipate perceptual requirements from long-term memories of past experiences and manage its computing resources for future expected stimuli. To study such mechanisms, I will build a framework for distributed perceptual processing at a large scale based on existing open-source technologies to combine the stimuli of multiple stationary and mobile robot-embedded sensors and balancing the resources of multiple computing systems. This will be done to investigate how a robot can build multimodal understandings of their surroundings. Then, an SA-inspired mechanism will also filter the percepts encoded in both working and long-term memories to avoid overloading computing resources. Finally, to go beyond reactive resource management and prepare for expected future events, an anticipatory supervision mechanism will be designed to infer which stimuli to expect from past experiences. Validation of this research program will be done with wheeled humanoid robots with advanced manipulation and sensing capabilities in realistic settings. Participating to an international competition such as RoboCup@Home is planned to serve as common ground for evaluation of the added value of the mechanisms developed in relation to state-of-the-art in the field. This research program represents a unique opportunity to study embodied artificial intelligence (AI) to respond and anticipate to events in real interactive environments with humans. The program will include multiple projects for MSc and PhD students to train them as experts in applied AI and robotics.
Program
Discovery Grants Program - Individual
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2019-06047
Name nom
Ferland, François
Department département
Faculté de génie
Organization id
68
Institution établissement
Université de Sherbrooke
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2019
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2606
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Systems, man and cybernetics
Research subject fr
Systèmes, homme et cybernétique
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Enabling rich, open-ended human-robot interaction through robust, advanced multimodal perceptual capabilities for high-level reasoning
Keywords
autonomous and intelligent systems; human-robot interaction; mobile robots; cognitive robotics; robot control architectures; machine learning; collaborative robotics; resource management; distributed software architectures; resource management
Application summary
The ultimate goal in Human-Robot Interaction is to have autonomous robots that can naturally interact with people, assist them in their daily lives at home or at work, and automatically adapt to new situations. Building such robots brings a great number of scientific challenges: safe, natural and effective interaction implies advanced perceptual capabilities to supply information for sound and robust reasoning. As humans operate in dynamic environments, robots will need a broad understanding of the world we live in and reasoning capabilities outside of the usually rigid, pre-defined scenarios they are currently programmed for. My research program is oriented toward enabling rich, open-ended interactions between humans and autonomous mobile robots. Doing so require high-level reasoning, which I believe is only possible with large-scale and multimodal perceptual capabilities on robots. To provide such capabilities, and as identified by studies on selective attention (SA) in neurosciences, there is growing evidence that top-down modulation occurs in the brain to maintain performance of the working memory by prioritizing already encoded items depending on the changing tasks demands and expectations of dynamic environments. A robot meant for rich interaction will have to adapt itself to such environments and prioritize what needs to be encoded in its own working memory. Furthermore, it will have to anticipate perceptual requirements from long-term memories of past experiences and manage its computing resources for future expected stimuli. To study such mechanisms, I will build a framework for distributed perceptual processing at a large scale based on existing open-source technologies to combine the stimuli of multiple stationary and mobile robot-embedded sensors and balancing the resources of multiple computing systems. This will be done to investigate how a robot can build multimodal understandings of their surroundings. Then, an SA-inspired mechanism will also filter the percepts encoded in both working and long-term memories to avoid overloading computing resources. Finally, to go beyond reactive resource management and prepare for expected future events, an anticipatory supervision mechanism will be designed to infer which stimuli to expect from past experiences. Validation of this research program will be done with wheeled humanoid robots with advanced manipulation and sensing capabilities in realistic settings. Participating to an international competition such as RoboCup@Home is planned to serve as common ground for evaluation of the added value of the mechanisms developed in relation to state-of-the-art in the field. This research program represents a unique opportunity to study embodied artificial intelligence (AI) to respond and anticipate to events in real interactive environments with humans. The program will include multiple projects for MSc and PhD students to train them as experts in applied AI and robotics.