Grant: Research Tools and Instruments

We request funding to purchase a panoptic (i.e. omnidirectional multi-camera) passive acquisition system capable of recording at 200Hz @ 12Mega-Pixels/camera, and a compute server capable of interfacing with such sensors and efficiently generating/processing 4D data (i.e.3D point clouds in motion). This capture system will allow us to generate high-quality 4D datasets. Consumer-level 4D sensors (e.g., Microsoft Kinect) are unsuitable, as they have limited resolution in both time (e.g., 30Hz) and space (e.g., .3Mega-Pixels). Furthermore, data acquired with consumer-level equipment contains substantial missing data, as no measurements are available on parts of an object facing away from the camera. Unfortunately, many of them cannot be arranged into a panoptic system due to interference and lack of shutter synchronization. The requested hardware overcomes the aforementioned challenges, and will allow us to generate high-quality datasets with minimal amounts of missing data. These will be used to train machine-learning models for motion-tracking, develop techniques for the creation of digital clones of non-rigid objects, and design algorithms that can understand object interactions in real-time by processing information from multiple vantage points. Researching ways to model and reason about 4D data is of paramount importance to general public. Technology that can understand the ever-changing environment can be used to create new forms of interaction and communication, and lie at the foundation of decision-making in autonomous robotic systems. In the era of big data and deep learning, the quality and size of training datasets is the key factor that allows quantum leaps in an algorithms performance. Without the ability to create new datasets, our ability to perform research in these domains will be severely hindered. Our research will acquire new datasets, and then leverage them to power algorithms. We will devise new non-intrusive methods to digitize the analog movements of our bodies to enable their use in navigating digital content, construct high-quality digital content that encode the dynamic nature of 3D objects, and develop techniques to non-intrusively capture object interactions at a level of complexity unseen before. The equipment will contribute to more than our own research programs. Not only will our datasets be publicly available to other researchers, but this equipment will also help us establish collaborations with other visual computing researchers across Canada. The personnel employing this equipment will positively contribute to the Canadian job market, having gathered experience in real-time computer vision, artificial intelligence, and content creation for the emerging AR/VR market.