Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Queen's University

$46,000.00

Unraveling Cognitive Strategies in Visual Search: Decoding Eye movement Patterns through Machine Learning Techniques

EF 2024-25

Résumé
In decision making research, agents have to choose between pursuing an ongoing strategy versus exploring a new one, as captured by the Exploration-Exploitation (E-E) framework. In the context of visual search, the framework provides a rich context in which to understand cognitive strategies: exploring new information vs. exploiting prior knowledge. For instance, when searching for your keys, you will check all likely locations (exploit strategy), before searching in unlikely places (explore strategy). Using the visual search task, we can explore how information accumulates over time and influences strategy shifts, providing an investigatory sandbox for strategy dynamics unconstrained by turn-taking found in decision tasks. With recent advancements in machine learning and computer vision, we have new avenues for analyzing eye movement patterns and develop new technology to capture these dynamics. In the current proposal, our team will apply our expertise on visual attention, visual search, scene perception, and eye movements to develop innovative machine learning models capable of extracting cognitive strategies (i.e., explore and exploitive) based on eye movement patterns. In addition, the proposal aims to investigate how machine learning models can be integrated with brain imaging techniques, such as electroencephalography (EEG) to explore neural correlates of cognitive strategies, using both Event-Related Potentials (ERPs for discrete processing differences) and Event-Related Spectral Perturbation (ERSP to measure cognitive effort and learning). In Project 1, we will create a number of eye movement datasets with different properties that would be used to develop a richer theoretical picture of how strategies develop, evolve and interact with different environment parameters. In Project 2, the eye movement datasets from Project 1 will be used to develop machine learning models of the eye movement patterns that can classify strategies and when individuals switch between strategies in real-time during a trial. In Project 3, we will examine both the Event-Related Potential (ERP) and Event-Related Spectral Perturbation (ERSP) related to each cognitive strategy and switches between them. The short-term goal of the proposed research is to develop a machine learning models that can classify the different strategy types (exploit and explore) in real-time and detect switches, as well as further theoretical understanding of the processing and neural underpinnings of cognitive strategies. In the long-term, the goal is to develop eye movement-based AI systems that can identify cognitive states across tasks. This will include not only whether a person is on task but also how they complete the task (easily or with difficulty), in order to develop more effective and adaptive training that can take into account the user's current cognitive state and incorporate individual differences in learning.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2024-04379
Name nom
Castelhano, Monica
Department département
Psychology
Organization id
29
Institution établissement
Queen's University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1502
Committee name en
Biological Systems and Functions
Committee name fr
Systèmes et fonctions biologiques
Area of application code
1210
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Psychology
Area of application fr
Psychologie
Research subject code
6307
Research subject group en
Psychology
Research subject group fr
Psychologie
Research subject en
Cognitive science -- other
Research subject fr
Science cognitive -- autres
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Unraveling Cognitive Strategies in Visual Search: Decoding Eye movement Patterns through Machine Learning Techniques
Keywords
eye movements; visual attention; decision making; statistical learning; scene processing; memory; cognitive science
Application summary
In decision making research, agents have to choose between pursuing an ongoing strategy versus exploring a new one, as captured by the Exploration-Exploitation (E-E) framework. In the context of visual search, the framework provides a rich context in which to understand cognitive strategies: exploring new information vs. exploiting prior knowledge. For instance, when searching for your keys, you will check all likely locations (exploit strategy), before searching in unlikely places (explore strategy). Using the visual search task, we can explore how information accumulates over time and influences strategy shifts, providing an investigatory sandbox for strategy dynamics unconstrained by turn-taking found in decision tasks. With recent advancements in machine learning and computer vision, we have new avenues for analyzing eye movement patterns and develop new technology to capture these dynamics. In the current proposal, our team will apply our expertise on visual attention, visual search, scene perception, and eye movements to develop innovative machine learning models capable of extracting cognitive strategies (i.e., explore and exploitive) based on eye movement patterns. In addition, the proposal aims to investigate how machine learning models can be integrated with brain imaging techniques, such as electroencephalography (EEG) to explore neural correlates of cognitive strategies, using both Event-Related Potentials (ERPs for discrete processing differences) and Event-Related Spectral Perturbation (ERSP to measure cognitive effort and learning). In Project 1, we will create a number of eye movement datasets with different properties that would be used to develop a richer theoretical picture of how strategies develop, evolve and interact with different environment parameters. In Project 2, the eye movement datasets from Project 1 will be used to develop machine learning models of the eye movement patterns that can classify strategies and when individuals switch between strategies in real-time during a trial. In Project 3, we will examine both the Event-Related Potential (ERP) and Event-Related Spectral Perturbation (ERSP) related to each cognitive strategy and switches between them. The short-term goal of the proposed research is to develop a machine learning models that can classify the different strategy types (exploit and explore) in real-time and detect switches, as well as further theoretical understanding of the processing and neural underpinnings of cognitive strategies. In the long-term, the goal is to develop eye movement-based AI systems that can identify cognitive states across tasks. This will include not only whether a person is on task but also how they complete the task (easily or with difficulty), in order to develop more effective and adaptive training that can take into account the user's current cognitive state and incorporate individual differences in learning.