Grant: Engage Grants Program

AWE Company Ltd. has been contracted to supply the technology for an Augmented/Mixed Reality tour of the entire grounds of Fort York historical site. They currently have the ability to create indoor AR experiences demonstrated by their success at creating a cutting-edge AR experience at Fort York. Expanding from a single indoor environment capable of five users to a highly dynamic outdoor AR experience capable of up to 60 users is a significant and necessary challenge for AWE. Their current solutions rely on an accurate magnetic tracking system used to determine the position and orientation of their mobile devices. This tracking technology is infeasible to expand to large outdoor spaces thus they require an alternate solution relying solely on on-board mobile sensors. With existing techniques and off-the-shelf components, image-based AR/MR systems do not work effectively in large outdoor spaces. The main concerns with these spaces are (1) highly dynamic lighting and appearance due to weather conditions and seasonal effects, (2) tourist attractions have tourists walking throughout the scene causing problems with camera pose tracking, and (3) robust localization in GPS-denied locations with large amounts of radio-frequency noise. Thus any solution to this problem should rely solely on computer vision techniques. In this project, we will develop an AR/MR solution that leverages and extends existing mapping/tracking algorithms in new ways to solve this real-world problem robustly. Advances in this area will not only benefit this project but will have wide application in all outdoor computer vision scenarios. We will be developing the first completely image-based system that merges mapping and AR in a real-world dynamic outdoor environment without the aid of GPS or RF based location on low-power mobile devices. Also, the tracking/localization system developed will be robust to both changes in appearance due to seasonal changes (i.e. snow in the winter, rainy, and bright summer days) and be able to track effectively in the presence of moving people. This will require the development of methods to adapt feature descriptions over time and methods to identify moving objects.