Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

Université de Sherbrooke

$24,000.00

Scene interpretation for sensory substitution and intelligent agents: a neurocomputational approach

FY 2020-21

Summary
The long-term goal of my research is the development of assistive technologies with sensory substitution devices. Thanks to the great plasticity of the brain and the densely interconnected cortical areas, it is possible to complement and substitute one sensory modality with another. However current sensory substitution systems are not able to analyze visual or auditory scenes as they are essentially describing the environment without interpretation. Therefore they overload the audition or the vision of the user which might be uncomfortable or unhealthy. Interpretation of auditory, visual and multimodal scenes is also of great need to the development of intelligent agents that interact with their environment and human beings. But actually scene interpretation for intelligent agents is more or less a limited description of scenes and restraints to very specific applications. Through this research program I want to push as much as possible the research in multimodal scene interpretation for the next 5 coming years. To maximize the impact and while doing the research, I will frequently transfer results to applications in sensory substitution (FQRNT équipe) and in artificial intelligence for interacting agents (CHIST-ERA IGLU). To minimize the environmental footprint, the design and realization of the algorithms will favour architectures with the lowest power consumption. To this end, I will focus during the last 2 years on implementations with networks of spiking neurons. I am restricting the research program to sources that are visual (they are moving sources of given shapes, texture earning of simple features to coarsely segregate in a first pass the multimodal objects inside the scene. 2) Perceptual learning to build multimodal complex features and representations that preserve the perceptive organization of characteristics (shapes, timbre, pitch); 3) Semantic characterization of objects; 4) Scene analysis and interpretation. 5) Low-power neuromorphic implementations. Sensory substitution has a very strong potential for "exercises" and rewiring of the brain along with development of new brain prosthesis. Intelligent agents that can perceive their environment through scene analysis is a corner stone to the new AI economy and industry. Also, there is a strong need to develop the Canadian industry of spiking neural networks and neuromorphic systems.
Program
Discovery Grants Program - Individual
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2017-06218
Name nom
ROUAT, Jean
Department département
GEGI
Organization id
68
Institution établissement
Université de Sherbrooke
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2020
Competition year année de concours
2017
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1206
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Engineering
Area of application fr
Génie
Research subject code
2510
Research subject group en
Electrical and electronic engineering
Research subject group fr
Génie électrique et électronique
Research subject en
Adaptive, learning and evolutionary systems
Research subject fr
Systèmes d'apprentissage adaptatifs et évolutifs
Installment
inst:4-5
Partie
2
Nb partie
5
Application title
Scene interpretation for sensory substitution and intelligent agents: a neurocomputational approach
Keyword
NSERC
Application summary
The long-term goal of my research is the development of assistive technologies with sensory substitution devices. Thanks to the great plasticity of the brain and the densely interconnected cortical areas, it is possible to complement and substitute one sensory modality with another. However current sensory substitution systems are not able to analyze visual or auditory scenes as they are essentially describing the environment without interpretation. Therefore they overload the audition or the vision of the user which might be uncomfortable or unhealthy. Interpretation of auditory, visual and multimodal scenes is also of great need to the development of intelligent agents that interact with their environment and human beings. But actually scene interpretation for intelligent agents is more or less a limited description of scenes and restraints to very specific applications. Through this research program I want to push as much as possible the research in multimodal scene interpretation for the next 5 coming years. To maximize the impact and while doing the research, I will frequently transfer results to applications in sensory substitution (FQRNT équipe) and in artificial intelligence for interacting agents (CHIST-ERA IGLU). To minimize the environmental footprint, the design and realization of the algorithms will favour architectures with the lowest power consumption. To this end, I will focus during the last 2 years on implementations with networks of spiking neurons. I am restricting the research program to sources that are visual (they are moving sources of given shapes, texture earning of simple features to coarsely segregate in a first pass the multimodal objects inside the scene. 2) Perceptual learning to build multimodal complex features and representations that preserve the perceptive organization of characteristics (shapes, timbre, pitch); 3) Semantic characterization of objects; 4) Scene analysis and interpretation. 5) Low-power neuromorphic implementations. Sensory substitution has a very strong potential for "exercises" and rewiring of the brain along with development of new brain prosthesis. Intelligent agents that can perceive their environment through scene analysis is a corner stone to the new AI economy and industry. Also, there is a strong need to develop the Canadian industry of spiking neural networks and neuromorphic systems.