Grant: Discovery Launch Supplement

With devices like Apple's Vision Pro and Meta's Oculus Quest bringing content off of the screen and into the space around us, spatial gesture input has the potential to overtake touchscreen gestures as the primary input technique of the future. This shift in input, paired with recent advances in optical sensing, presents a unique opportunity to be at the forefront of designing and implementing spatial gestural interactions. The overarching vision of this program is to address a series of challenges surrounding spatial gesture input, in an effort to help facilitate lay users' transition to using spatial computing interfaces. To ensure these spatial gestures are useful to users, the first objective of my program is to identify which specific contexts can benefit from spatial gestures and determine best practices for their use in each context. A broad category that can leverage spatial gestures is extended reality (mixed, augmented, or virtual reality), where digital content is presented in free space around the user. These gestures may also have use in other emerging technologies where interaction is required at a distance, such as smart TVs, or where visual encumbrance limits precise targeting on a touchscreen, such as automotive infotainment systems. Within this objective, we will design gesture sets for several of these scenarios, evaluate them in their respective contexts, and deliver spatial gesture design considerations to assist developers of this technology. A fundamental problem with gestural input, is that often gestures are unintuitive and not discoverable. The proposed research program will aim to both bridge the gap in discoverability and reduce the learning curve for adoption of spatial gestures, particularly within the transitory period from predominantly touch based input. For instance, spatial gestures can be designed to be metaphorically synonymous with existing touch gestures or feed-forward guidance mechanisms can subtly nudge users into providing gesture input more efficiently. Despite significant advances in optical sensing approaches used to recognize spatial gestures, these systems will inevitably make errors in which they fail to capture or properly interpret user input. For instance, did the user gesture to provide input, or was it simply natural gesticulation during speech? Thus, when designing these gestural input systems we need to consider ideal characteristics of errors when they do occur, as well as elegant strategies for recovery and impact mitigation; which this program will address. In studying these aspects of spatial gestures, the outlined research program will prepare students for research and development careers within the tech industry, as well as provide them with specialized skills in engineering AI driven technology and extended reality applications. The program has the potential benefit of shaping how we will provide input to spatial computing environments, positioning Canada as a leader in the area.