Grant: Discovery Grants Program - Individual

Virtual, Mixed and Augmented Reality systems, collectively known as Extended Reality Systems (XRS) combine elements from virtual reality and the real environment around a user. XRS are used in simulated training, 3D telepresence, telehealth, and games, among other areas. The overall goal of this research is to improve the quality of the user experience through the development of advanced algorithms, innovative interaction devices, and software systems that will facilitate access to, and use of, fully mobile XRS by a wider segment of the population. We want to bring forward the next generation of mobile XRS through the research and development of advanced systems where users enjoy a variety of interaction styles while retaining freedom of movement and posture. This shall move us closer to the type of interfaces that we aspire to have when using XRS, ones which are convenient, easy to use, and intuitive. We will focus on the following objectives: 1. To address challenges in the integration of XRS with mobile devices: The focus in this theme is the development of algorithms to improve the integration of smartphones and tablets with XRS to allow people to be aware of- and interact with their surroundings, and to communicate with friends, colleagues, and others. This integration will allow XRS users to experience a combination of the virtual and real worlds in a way that does not isolate them from the real world, but rather enhances it. We plan to develop better algorithms for closer integration between mobile devices and XRS, so that people can experience XRS outside of the lab environment. 2. To develop new algorithms and tangible interfaces for natural, intuitive, and flexible mobile systems in Mixed Reality contexts: We want to develop new algorithms and tangible interfaces for natural, intuitive and flexible mobile systems in Mixed Reality contexts. Near-Range Augmented Virtuality, i.e. the overlay of virtual imagery on the real world for objects in close proximity, provides opportunities to interact with both the physical world and the virtual world simultaneously. Through collaboration with the Office of Professional and Educational Development at Memorial, we plan to continue exploring ways to use XRS for telepresence for remote training. 3. To improve telepresence systems that use Re-projected Avatars to bring people together in ways not possible until now: People can benefit from mobile Mixed Reality systems (MXRs) for Telepresence if we manage to produce XRS that are socially acceptable and perceptually transparent. Re-projected Avatars in XRS would allow participants of a 3D teleconference system to see the other party at a more convenient distance than what is possible in the real world. XRS are the ideal technologies to do this. Thus, research is needed to apply these concepts properly and to study the human factors that would need to be considered to achieve positive results using these new forms of interaction.