Grant: Discovery Grants Program - Individual
Université de Montréal
$38,000.00
Individual differences in natural scene semantics investigated with machine learning and neuroimaging
FY 2021-22
Summary
The ability to make sense of our visual environment is crucial for adaptive behaviour and survival. Yet, precisely how this subjective, conscious experience of the world emerges in our brain remains poorly understood. The long term aim of my research programme is to provide a new way of thinking about the visual system, placing emphasis on semantic representations and individual differences in the context of natural scene processing. Doing so, we will fill an important gap in our understanding of visual recognition by elucidating the neural underpinnings of semantic processing and its relationship with visual consciousness and subjective experience. How does the brain transform visual information from the retina into high-level semantic representations enabling us to clearly and precisely communicate what we see? Classic accounts suggest functional segregation in two distinct cortical pathways: a "where" and a "what" pathway, a dorsal stream specialised in spatial information ("where") and a ventral stream for category or conceptual information. The ventral stream could therefore be seen as a distributed system where overlapping feature maps encode specific dimensions about particular objects. This view might stem from simple experimental paradigms involving single objects or simplified stimuli, but it is not clear whether this view accurately generalises to naturalistic viewing situations with complex scenes containing multiple objects and concepts. Could there be higher-level (linguistic and semantic conscious form) features, being integrated over object and scene feature maps to provide meaning? To answer these questions, we will combine brain activity measured from electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) while participants view rich and complex natural scenes, with artificial neural networks (ANNs) trained on sentence embeddings, and a battery of innovative behavioural tasks (collected using my online platform for behavioural experiments https://meadows-research.com). The focus for this DG cycle is on moving theories of visual representations into the domains of complex naturalistic stimuli, semantic representations, and individual differences, all of which are relatively unexplored territories in the field. This will be achieved with three core aims: 1- Precisely track where in the brain and when in time after viewing a natural scene, semantic representations are established. 2- Reveal how activity patterns related to scene semantics influence the content of our visual consciousness. 3- Identify links between fine-scale differences in brain representations (i.e. the variations in activity pattern discriminability between individuals) and behaviour (measured in an array of cognitive tasks).
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2021-03127
Name nom
Charest, Ian
Department département
Psychologie
Organization id
63
Institution établissement
Université de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2021
Award amount
38000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1502
Committee name en
Biological Systems and Functions
Committee name fr
Systèmes et fonctions biologiques
Area of application code
1210
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Psychology
Area of application fr
Psychologie
Research subject code
6300
Research subject group en
Psychology
Research subject group fr
Psychologie
Research subject en
Psychology
Research subject fr
Psychologie
Installment
inst:1
Partie
1
Nb partie
5
Application title
Individual differences in natural scene semantics investigated with machine learning and neuroimaging
Keyword
NSERC
Application summary
The ability to make sense of our visual environment is crucial for adaptive behaviour and survival. Yet, precisely how this subjective, conscious experience of the world emerges in our brain remains poorly understood. The long term aim of my research programme is to provide a new way of thinking about the visual system, placing emphasis on semantic representations and individual differences in the context of natural scene processing. Doing so, we will fill an important gap in our understanding of visual recognition by elucidating the neural underpinnings of semantic processing and its relationship with visual consciousness and subjective experience. How does the brain transform visual information from the retina into high-level semantic representations enabling us to clearly and precisely communicate what we see? Classic accounts suggest functional segregation in two distinct cortical pathways: a "where" and a "what" pathway, a dorsal stream specialised in spatial information ("where") and a ventral stream for category or conceptual information. The ventral stream could therefore be seen as a distributed system where overlapping feature maps encode specific dimensions about particular objects. This view might stem from simple experimental paradigms involving single objects or simplified stimuli, but it is not clear whether this view accurately generalises to naturalistic viewing situations with complex scenes containing multiple objects and concepts. Could there be higher-level (linguistic and semantic conscious form) features, being integrated over object and scene feature maps to provide meaning? To answer these questions, we will combine brain activity measured from electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) while participants view rich and complex natural scenes, with artificial neural networks (ANNs) trained on sentence embeddings, and a battery of innovative behavioural tasks (collected using my online platform for behavioural experiments https://meadows-research.com). The focus for this DG cycle is on moving theories of visual representations into the domains of complex naturalistic stimuli, semantic representations, and individual differences, all of which are relatively unexplored territories in the field. This will be achieved with three core aims: 1- Precisely track where in the brain and when in time after viewing a natural scene, semantic representations are established. 2- Reveal how activity patterns related to scene semantics influence the content of our visual consciousness. 3- Identify links between fine-scale differences in brain representations (i.e. the variations in activity pattern discriminability between individuals) and behaviour (measured in an array of cognitive tasks).