Grant: Discovery Grants Program - Individual

Mixed Reality Groupware to Support Physical Collaborations Physical collaborations, where people work together to move, manipulate, build, and repair physical objects, are extremely common activities. Whether building a digital circuit or repairing an engine, playing a game or following an exercise, people work together to share ideas and receive guidance. Mixed reality systems are well-suited to support physical collaborations, since they allow virtual objects and other useful information to be displayed alongside physical tasks, which makes providing and receiving guidance through technology easier and more natural. The problem is that we do not know how to design mixed reality-based groupware (MR-Groupware) to best support collaboration in physical tasks when collaborators are separated by time or distance. For example, providing guidance to a friend on how to repair a leaky faucet using current video conferencing technology is slow and cumbersome: its difficult for the remote helper to describe and demonstrate the movements to be made, and its difficult for the person being guided to see and understand what the helper wants them to do. When we are separated from others there are two primary reasons that difficulties arise. First, we lack an awareness of all of the rich information we are able to take in when we are working with someone face-to-face (e.g., what is the other person doing, what are they looking at, where did they put the tools, etc). These awareness questions are tough to gather when using current communication technology. Second, we lack the embodiments that come from being physically collocated with someone, and that make face-to-face collaborations easy. For example, it is tough to demonstrate to someone how to use a wrench when the tool is thousands of kilometers away with a collaborator who is trying to fix the faucet. This research program will address the two key challenges (awareness and embodiment) for working with a separated collaborator through MR-Groupware by developing new interactions and technologies. My long-term research objective is to allow remote or time-separated collaboration with others on physical tasks to be as natural as having them stand beside you. This research program will make substantial progress towards this objective through a highly collaborative plan, which leverages our already established strategic relationships with leading mixed reality companies and academic partners. To maximize the impact of our work, we will produce new open-source software toolkits that allow MR-Groupware to be rapidly matured. MR-Groupware is a strategic technology for Canada, since it has already been successfully applied to a wide range of strategic industries (e.g., aeropsace, defence, training/education, video games, and health). The mixed reality industry in Canada is rapidly growing, and the HQP trained through my research program will be well positioned to make important contributions to this new field.