Grant: Discovery Grants Program - Individual
The University of British Columbia
$39,000.00
Interacting with Augmented Reality
FY 2023-24
Summary
Computing technology is now ubiquitous, with the digital environment offering an ever-expanding array of services and conveniences. However, computers and their digital contents have remained largely separated from the real, physical world - users explicitly interact with computers through various screens and input devices, leading to a gap between the physical and digital worlds. The rise of immersive augmented reality points towards a future in which the lines between physical and virtual will increasingly blur, and the entire physical environment will become a rich canvas for digital content displayed via projection, head-mounted displays, or even smart glasses. Yet, there is much work to be done to achieve a future in which digital content can live so harmoniously alongside physical reality, instead of merely inside of it. I propose to investigate how we would interact with real-world digital content. My research will leverage new developments in computing and interface technology, including the miniaturization of high-performance computational platforms, recent advances in projection technologies and augmented reality, and the use of machine learning and artificial intelligence to learn and predict user behaviour, intent and input - all areas in which I have made substantial prior contributions. I will explore interaction techniques through three objectives: robustly sensing input, designing practical interactions and deploying integrated systems. To more robustly sense input, I propose exploring various non-traditional sensory channels - including electromagnetic sensing, Doppler radar, acoustic and vibration sensing - as potential avenues for understanding the actions and behaviour of users. Simultaneously, I will improve the robustness of existing sensing systems by applying recent advances in deep learning to the signal processing problem, as well as applying targeted reductions to latency and error rate through clever hardware and software design. To design practical interactions, I propose to explore the design space of on-world user interactions, enabling users to use the full capabilities of augmented reality systems, and using the discovered interactions to design more targeted input sensing approaches. Finally, I will explore a real-world deployment of the technology, designing an integrated platform which can serve as a base for further high-level explorations into this topic. I propose exploring the feasibility of transforming the ordinary light bulb into a kind of computational "info-bulb", combining projection, sensing, and computational capabilities into a device that fits into standard light fixtures, able to project interactive interfaces onto the environment and paving the way for ubiquitous augmented reality.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2019-05624
Name nom
Xiao, Robert
Department département
Computer Science
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
39000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
800
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information and communication services
Area of application fr
Services d'information et de télécommunications
Research subject code
2710
Research subject group en
Information technology
Research subject group fr
Information systems design
Research subject en
Technologie de l'information
Research subject fr
Conception de systèmes d'information
Application title
Interacting with Augmented Reality
Keyword
Augmented Reality; Human-Computer Interaction; On-World Computing; User Interfaces
Application summary
Computing technology is now ubiquitous, with the digital environment offering an ever-expanding array of services and conveniences. However, computers and their digital contents have remained largely separated from the real, physical world - users explicitly interact with computers through various screens and input devices, leading to a gap between the physical and digital worlds. The rise of immersive augmented reality points towards a future in which the lines between physical and virtual will increasingly blur, and the entire physical environment will become a rich canvas for digital content displayed via projection, head-mounted displays, or even smart glasses. Yet, there is much work to be done to achieve a future in which digital content can live so harmoniously alongside physical reality, instead of merely inside of it. I propose to investigate how we would interact with real-world digital content. My research will leverage new developments in computing and interface technology, including the miniaturization of high-performance computational platforms, recent advances in projection technologies and augmented reality, and the use of machine learning and artificial intelligence to learn and predict user behaviour, intent and input - all areas in which I have made substantial prior contributions. I will explore interaction techniques through three objectives: robustly sensing input, designing practical interactions and deploying integrated systems. To more robustly sense input, I propose exploring various non-traditional sensory channels - including electromagnetic sensing, Doppler radar, acoustic and vibration sensing - as potential avenues for understanding the actions and behaviour of users. Simultaneously, I will improve the robustness of existing sensing systems by applying recent advances in deep learning to the signal processing problem, as well as applying targeted reductions to latency and error rate through clever hardware and software design. To design practical interactions, I propose to explore the design space of on-world user interactions, enabling users to use the full capabilities of augmented reality systems, and using the discovered interactions to design more targeted input sensing approaches. Finally, I will explore a real-world deployment of the technology, designing an integrated platform which can serve as a base for further high-level explorations into this topic. I propose exploring the feasibility of transforming the ordinary light bulb into a kind of computational "info-bulb", combining projection, sensing, and computational capabilities into a device that fits into standard light fixtures, able to project interactive interfaces onto the environment and paving the way for ubiquitous augmented reality.