Grant: Discovery Grants Program - Individual

The overarching goal of my research is to understand how the human brain uses vision to perceive the world and perform actions. The short-term goal for this granting period is to cultivate a new approach that I call Immersive Neuroscience to use novel technologies to study the brain and behavior in the real world and compelling simulations. Over my past three NSERC Discovery grants, I have developed techniques to study real actions and real objects using human neuroimaging (specifically functional magnetic resonance imaging, fMRI). This research suggests that real stimuli and real actions are more engaging -- both behaviorally and neurally -- than simulations (such as pantomimed actions and images), a phenomenon we call the "real-world advantage". While the results have been suggestive, using fMRI to study real situations is very challenging and has limitations. Emerging technologies are providing new ways forward for Immersive Neuroscience. First, 3D displays and virtual/augmented reality technologies enable the presentation of compelling real-world scenes, enabling cognitive neuroscience to move beyond heavy reliance on static 2D images. Second, new optical neuroimaging technologies (functional near infrared spectroscopy, fNIRS, and high-resolution approaches like the Kernel Flow) enable the measurement of cortical brain responses directly in the real world. My research program will pioneer new approaches with these technologies to bring cognitive neuroscience research closer to the real world. This grant will focus on the following three lines of research. First, I will examine how the human brain processes complex three-dimensional visual stimuli like faces, scenes and objects. Second, I will examine which aspects of realism (particularly 3D vision and actability) change behavior and brain responses to real objects and real people compared to images or videos. Third, I will examine how the brain functions during realistic tasks in which participants actively control their environments and get ongoing feedback. Immersive Neuroscience will provide rich opportunities for next-generation neuroscience to move closer to everyday life by pushing the realism of cognitive neuroscience while maintaining sufficient control for carefully designed experiments. This approach offers the potential to make innovative discoveries about the factors crucial to realism. Moreover, it enhances the likelihood of success for applications of virtual reality (e.g., training, image-guided surgery) and optical neuroimaging (e.g., brain-computer interfaces).