Grant: Discovery Grants Program - Individual

Vision-based systems are a central research topic in computing, robotics and artificial intelligence. This prominence arises as current trends in automotive, consumer and home technology (e.g., Amazon Echo, Google Home, Apple HomePod) are leveraging video data to understand and react to the environment. Moreover, interesting scientific questions arise as we seek to understand how it is possible to recover detailed information about the world from a mere captured video. My research targets the underpinning theoretical and engineering principles of video understanding.My particular studies of visual information provide insights into the fundamental properties of temporal sequences of images (e.g., video) and extend our technological capabilities by indicating novel applications. I use mathematical techniques to analyze images, develop processing algorithms based on the resulting analyses and carry out empirical tests to evaluate their performance. This approach allows us to ask and answer basic questions about video and other sources of temporal image sequences. Questions of interest include: How can complex streams of video be organized, archived and searched on the basis of visual content? How can multiple views of a captured scene be used to reconstruct a three-dimensional model of the scene? Key novelty comes about via an uncommon level of explicit analytic detail in system specification, which provides benefits of clarity in how the system operates and greatly decreased need for training data in comparison to most other systems.My research contributes to Canada in three important ways. First, it advances the discipline of computer vision as we expose basic properties of image information and develop corresponding algorithms for incorporation into vision machines; this advance allows Canada to compete successfully in a critical area of science and technology. Second, it provides students with hands on experience and skills in vital areas of investigation: computer vision, video processing and, more generally, computer science and engineering. Students go on to make careers in science, technology, teaching and elsewhere, taking with them the enrichment of having been involved in research. Third, it paves the way for novel ways to exploit image data in practice. Applications of interest include video processing for everyday concerns ranging from the internet to personal handheld devices, video enhancement so that content is more readily interpretable and intelligent processing modules for robots and autonomous vehicles. The work thereby benefits both producers as well as consumers of such technologies and services.