Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Toronto

$110,000.00

Active Robotic Vision: Dynamic Sensor Systems for Informed Robotic Perception

EF 2023-24

Résumé
Increasingly, robotic systems employ large numbers of sensors to perceive their environment in all directions. These static arrays with fixed viewpoints are expensive, require large computing power, and have limited ability to adapt to changing task requirements. In this program, I will develop active perception methods for dynamic sensor systems, which use actuated sensors to gain viewpoint flexibility and reduce computing and cost requirements for robots. Dynamic sensor systems add two complexities to typical robotic perception. First, careful calibration is required to fuse actuated and static sensor data. This includes the kinematic parameters of actuators and the relative pose estimation between each sensor and the robot base frame, which vary during operation. Second, methods for the active selection of sensor viewpoints are needed to predict the importance of future measurements to the perception task at hand. I will focus on simultaneous localization and mapping throughout this program, as a canonical example of an active perception task, but the methods developed can generalize to other domains such as object tracking, grasping, 3D reconstruction and inspection. The program includes three main objectives geared toward unlocking the potential of dynamic sensor systems. I will develop novel calibration methods based on factor graph optimization that simultaneously resolve robot motion through an unknown environment and identify temporal, kinematic and extrinsic calibration for static and dynamic sensors. These methods will unify the calibration process for dynamic sensor systems and remove the current reliance on specially design targets. I will then develop active perception methods based on information gain prediction that inform a greedy viewpoint selection strategy. These predictions will be based on the existing map and localization uncertainty, and will evaluate the tradeoff between refining the current map state and exploring new areas to reach the desired goal location. Finally, I will refine information gain prediction through hybrid recurrent neural network training and apply reinforcement learning to produce viewpoints directly in an end-to-end approach. As information gain prediction is challenging to model precisely, training deep networks over varied environments, paths, and sensor viewpoints will lead to improved prediction performance. This work will enable robotic systems with actuated sensors and manipulators to better achieve their objectives, and will lead to a deeper understanding of the connections between information gain, viewpoint selection and localization and mapping robustness. Drones, ground vehicles and mobile manipulators will then be able to operate in more complex spaces safely, perform detailed inspections of aging infrastructure, produce accurate 3D models of objects for grasping and manipulation, and even operate in close proximity to humans through improved situational awareness.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2019-05939
Name nom
Waslander, Steven
Department département
Aerospace Studies, Institute for - Aerospace Studies, Institute for
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
110000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
703
Area of application group en
Transportation systems and services
Area of application group fr
Services et systèmes de transport
Area of application en
Aerospace
Area of application fr
Aérospatial
Research subject code
2605
Research subject group en
Robotics
Research subject group fr
Pattern analysis and machine intelligence
Research subject en
Robotique
Research subject fr
Analyse de formes et intelligence artificielle
Application title
Active Robotic Vision: Dynamic Sensor Systems for Informed Robotic Perception
Keyword
calibration; gimballed cameras; information theory; multi-camera systems; recurrent neural networks; reinforcement learning; robotics; sensor fusion; unmanned aerial vehicles; visual perception
Application summary
Increasingly, robotic systems employ large numbers of sensors to perceive their environment in all directions. These static arrays with fixed viewpoints are expensive, require large computing power, and have limited ability to adapt to changing task requirements. In this program, I will develop active perception methods for dynamic sensor systems, which use actuated sensors to gain viewpoint flexibility and reduce computing and cost requirements for robots. Dynamic sensor systems add two complexities to typical robotic perception. First, careful calibration is required to fuse actuated and static sensor data. This includes the kinematic parameters of actuators and the relative pose estimation between each sensor and the robot base frame, which vary during operation. Second, methods for the active selection of sensor viewpoints are needed to predict the importance of future measurements to the perception task at hand. I will focus on simultaneous localization and mapping throughout this program, as a canonical example of an active perception task, but the methods developed can generalize to other domains such as object tracking, grasping, 3D reconstruction and inspection. The program includes three main objectives geared toward unlocking the potential of dynamic sensor systems. I will develop novel calibration methods based on factor graph optimization that simultaneously resolve robot motion through an unknown environment and identify temporal, kinematic and extrinsic calibration for static and dynamic sensors. These methods will unify the calibration process for dynamic sensor systems and remove the current reliance on specially design targets. I will then develop active perception methods based on information gain prediction that inform a greedy viewpoint selection strategy. These predictions will be based on the existing map and localization uncertainty, and will evaluate the tradeoff between refining the current map state and exploring new areas to reach the desired goal location. Finally, I will refine information gain prediction through hybrid recurrent neural network training and apply reinforcement learning to produce viewpoints directly in an end-to-end approach. As information gain prediction is challenging to model precisely, training deep networks over varied environments, paths, and sensor viewpoints will lead to improved prediction performance. This work will enable robotic systems with actuated sensors and manipulators to better achieve their objectives, and will lead to a deeper understanding of the connections between information gain, viewpoint selection and localization and mapping robustness. Drones, ground vehicles and mobile manipulators will then be able to operate in more complex spaces safely, perform detailed inspections of aging infrastructure, produce accurate 3D models of objects for grasping and manipulation, and even operate in close proximity to humans through improved situational awareness.