Grant: Discovery Grants Program - Individual
Simon Fraser University
$54,000.00
SFM++: Toward Active, Collaborative, and Semantic Structure-from-Motion (SFM)
EF 2014-15
Résumé
Structure-from-Motion (SFM) techniques recover 3D information from 2D images. SFM has many applications, including 3D sensing in robotics and 3D content creation in computer games or movies. This proposal studies SFM from a unique perspective. I will take input from multiple independently moving video cameras with wide baselines to address 3D vision challenges. In comparison, previous methods often employ either a single camera or an array of fixed cameras with short baselines. These moving cameras can be handheld camcorders or wireless cameras mounted on different robot platforms. SFM in this novel setting is largely unexplored except in a few of our recent works [3, 4]. As demonstrated in these two works, this new SFM technique can be potentially used to guide a robot team for vision-based autonomous navigation or to provide a flexible handheld camcorder-based 3D reconstruction solution. This study has the potential to make a profound impact. It is the first attempt to develop a collaborative 3D vision system for a team of miniature robots. Robot vision is possibly the most important application of computer vision. Vision powered autonomous robots will bring revolutionary changes to society and benefit everybody. Due to the difficulties in computer vision, existing robot platforms often rely on bulky laser or ultrasonic scanners to solve the 3D sensing problem. However, these sensors are too heavy or energy inefficient for a miniature platform, e.g. a miniature robot with a payload of less than 50g. Vision is the ideal solution for 3D sensing on these miniature robots. Thus, the robotics and computer vision community devote tremendous efforts to designing 3D vision systems for robots. Almost all of these works focus on a single robot. To tackle difficult 3D vision problems, this proposal studies the collaborative vision sensing of a robot team. By exploiting the collaboration across multiple cameras, my students and I aim to develop robust and efficient computational algorithms for various 3D vision tasks. Specifically, we will address the robot team exploration problem to actively plan their navigation path to enhance 3D sensing. This includes maintaining view overlap between different robots and reducing measurement uncertainty. We will further develop real-time dense 3D reconstruction algorithms from multiple moving cameras, which can recover per-pixel depth information even under complicated dynamic environments. It effectively turns cameras into 3D scanners. We further plan to estimate skeletal human motion from the recovered 3D data by the collaboration of multiple cameras. This proposal will enable a team of miniature robots to navigate autonomously in complicated environments with moving objects. The robot team can sense skeletal human motion which enables natural human robot interaction. We will integrate and demonstrate our results with existing toy or commercial miniature unmanned aerial vehicles (UAVs) such as AR.Drone and Pelican. These algorithms can also be used for applications in computer graphics, such as flexible motion capture with handheld camcorders. This program is globally unique. The 3D vision of a collaborative robot team is a largely unexplored field. It will clearly extend Canada?s lead in this area. Its results can be used in robotics applications for security/surveillance. The research result on 3D skeletal capture from handheld videos can be applied for flexible motion capture in the computer gaming industry, a major employer in Canada. Three PhD students will be trained in this project.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2014-06686
Name nom
Tan, Ping
Department département
Computer Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2014
Competition year année de concours
2014
Award amount
54000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Partie
1
Nb partie
5
Application title
SFM++: Toward Active, Collaborative, and Semantic Structure-from-Motion (SFM)
Keyword
NSERC
Application summary
Structure-from-Motion (SFM) techniques recover 3D information from 2D images. SFM has many applications, including 3D sensing in robotics and 3D content creation in computer games or movies. This proposal studies SFM from a unique perspective. I will take input from multiple independently moving video cameras with wide baselines to address 3D vision challenges. In comparison, previous methods often employ either a single camera or an array of fixed cameras with short baselines. These moving cameras can be handheld camcorders or wireless cameras mounted on different robot platforms. SFM in this novel setting is largely unexplored except in a few of our recent works [3, 4]. As demonstrated in these two works, this new SFM technique can be potentially used to guide a robot team for vision-based autonomous navigation or to provide a flexible handheld camcorder-based 3D reconstruction solution. This study has the potential to make a profound impact. It is the first attempt to develop a collaborative 3D vision system for a team of miniature robots. Robot vision is possibly the most important application of computer vision. Vision powered autonomous robots will bring revolutionary changes to society and benefit everybody. Due to the difficulties in computer vision, existing robot platforms often rely on bulky laser or ultrasonic scanners to solve the 3D sensing problem. However, these sensors are too heavy or energy inefficient for a miniature platform, e.g. a miniature robot with a payload of less than 50g. Vision is the ideal solution for 3D sensing on these miniature robots. Thus, the robotics and computer vision community devote tremendous efforts to designing 3D vision systems for robots. Almost all of these works focus on a single robot. To tackle difficult 3D vision problems, this proposal studies the collaborative vision sensing of a robot team. By exploiting the collaboration across multiple cameras, my students and I aim to develop robust and efficient computational algorithms for various 3D vision tasks. Specifically, we will address the robot team exploration problem to actively plan their navigation path to enhance 3D sensing. This includes maintaining view overlap between different robots and reducing measurement uncertainty. We will further develop real-time dense 3D reconstruction algorithms from multiple moving cameras, which can recover per-pixel depth information even under complicated dynamic environments. It effectively turns cameras into 3D scanners. We further plan to estimate skeletal human motion from the recovered 3D data by the collaboration of multiple cameras. This proposal will enable a team of miniature robots to navigate autonomously in complicated environments with moving objects. The robot team can sense skeletal human motion which enables natural human robot interaction. We will integrate and demonstrate our results with existing toy or commercial miniature unmanned aerial vehicles (UAVs) such as AR.Drone and Pelican. These algorithms can also be used for applications in computer graphics, such as flexible motion capture with handheld camcorders. This program is globally unique. The 3D vision of a collaborative robot team is a largely unexplored field. It will clearly extend Canada?s lead in this area. Its results can be used in robotics applications for security/surveillance. The research result on 3D skeletal capture from handheld videos can be applied for flexible motion capture in the computer gaming industry, a major employer in Canada. Three PhD students will be trained in this project.