Grant: Research Tools and Instruments

Growing evidence suggests that the realism of stimuli and actions toward them affects behavior and brain processing. We are requesting equipment to: (1) project compelling binocular 3D displays during functional magnetic resonance imaging (fMRI) at 3 and 7 Tesla; (2) measure eye movements during fMRI to enable more realistic gaze strategies; and (3) record high-resolution video of hand movements and other actions in the scanner. This equipment will advance our research in four ways. First, the 3D projector will enable us to build upon recent and ongoing research showing differences in brain activation and patterns for real objects and real actions vs. common proxies (images and simulated actions). Specifically, it will allow us to determine which aspects of stimuli and tasks are essential to make something feel real to the participant and be treated as equivalent to real by the brain. Second, highly compelling displays of objects and scenes in the scanner will form part of a collaborative research program to develop new paradigms to study intersubject correlations of brain activation using active tasks in virtual environments that engage a broad range of sensory, cognitive and motor processes. Third, the ability to track eye movements binocularly inside the scanner will enable us to examine correlations between brain activation or patterns and gaze location rather than restricting gaze unnaturally, as is typical in fMRI studies. Fourth, the ability to record high-fidelity video in the scanner will enable us to characterize hand actions and examine correlations between kinematics and behavior. Taken together, these projects will move cognitive neuroscience forward by enhancing the realism of paradigms and ensuring that the virtual aspects engage the same neural systems as true reality. The projects bring together a team of researchers with expertise in visual perception and action, enhancing realism of research in psychology and neuroscience, adapting VR/AR for practical applications, and using fMRI and behavioral studies to understand cognitive functions with cutting-edge approaches. The topic is particularly timely given the rapid growth of virtual reality (VR) and augmented reality (AR) applications for simulator training, image-guided surgery, and rehabilitation as well as consumer applications on phones and tablets. Our work will inform technological developments and provide a demand for trainees to work in or liaise with industry.