Grant: Discovery Grants Program - Individual

There is a growing need for accessing information while carrying out a physical activity. This could be as simple as following a recipe while cooking, browsing ideas on Pinterest while landscaping, or more complex tasks such as being guided during engine repair or viewing stepwise instructions for resuscitating a stranger. Common among all these activities, is the need to peruse information in the midst of a task, or in-situ interaction. Current computing platforms do not adequately provide in-situ information access. The pervasive smartphone has a limited display, requiring significant attentional resources that could instead be used on the primary activity. Furthermore, such devices limit the use of hands for other tasks, as users need to hold the device for interaction. As a result, most adaptations of current devices for in-situ activity result in awkward physical configurations, which in some cases are life-threatening (e.g., browsing content or texting while walking). Emerging technologies, such as head-worn displays, smartwatches or smart-textiles, although not mature yet, offer an early platform for exploring the key requirements for developing design guidelines for in-situ user interfaces. Ideally such interfaces should integrate information into the user's vicinity, providing a larger visual canvas than that available on current mobile devices, thereby reducing split attention between content and task, and designed to ease information interaction. However, much work is needed to cultivate in such technologies support for in-situ information access. We frame our short-term goals in terms of four key requirements needed to engender such types of interactions: Content, Presentation, Interaction and Social Requirements. We address Content requirements through means of repurposing existing media, including instructional videos. Presentation requirements are met through smart visual embeddings, that intelligently intermesh content in the user's field-of-activity, taking into account saliency and supporting users' spatial memory. Interaction requirements build off sensory information that can interleave content with user activity. We address Social requirements, including social inclusion, through means of detecting salient interactions of co-located users. The growing importance of integrating information acquisition into our busy lives suggests strongly that in-situ user interfaces will become a standard computing paradigm. The proposed program will lead to training 15 HQP in 8 projects across the above four key axes. Pioneering advances will be made in Human-Computer Interaction (HCI), Personal and Immersive Visual Analytics, and Ubiquitous Computing. Developing this next generation computing paradigm will enable us to maintain a leadership position in fields of growing importance to the Canadian wearables and smart-textiles industries, and help Canadians by developing technologies that can directly improve their social well-being.