Grant: Collaborative Research and Development Grants

The human brain has evolved an impressive ability to integrate visual information with input from multiple sensory systems (e.g. vestibular, auditory). As a result, we seemingly effortlessly interact with, and navigate within, our environment. We naturally adapt to some changes in the relation between these inputs, for instance as the body grows, and also to more temporary changes in the correspondence between sensory information. The study of sensory integration has long been a focus of scientific study, using carefully controlled stimuli. However, it is difficult to extrapolate these results to vision in more natural environments. The rapid expansion of virtual reality (VR) and augmented reality (AR) content and displays provides opportunities to assess sensory integration in a controlled manner, but with more complex naturalistic stimuli. In addition, behavioural experiments have the potential to provide important insights into the impact of sensory distortions on user well-being and for task performance in virtual environments. In collaboration with Qualcomm Canada Inc., we will systematically evaluate the effects of sensory discord in virtual environments using rigorous behavioural assessment methods when users a) passively integrate conflicting visual spatial and geometric information, b) actively integrate conflicting visual depth/location cues and c) actively interpret self-motion information based on visual, auditory and vestibular cues. In all cases, we will use a combination of psychophysical methods, validated questionnaires and human factors protocols to evaluate the impact of the sensory distortions. Thus, the results of the proposed experiments will inform both our understanding of human visual processing under naturalistic viewing scenarios, and the user experience in advanced head-mounted displays (HMD).