Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Alberta

$54,000.00

Robot arm and hand manipulation using vision-guided motion control directed by human gestures

EF 2019-20

Résumé
This research will focus on the geometric specification of robot (arm and hand) manipulation tasks in camera (and other sensory) coordinate frames. We propose to develop Human-Robot Interfaces (HRI) based on pointing and gestures. The gestures will be interpreted through computer vision and robot motions will be performed through visual servoing, and force servoing. The main objective is to solve general robot arm and hand manipulation tasks interactively with humans.******Robotics research in both engineering and artificial intelligence has focused on autonomy. While autonomy works well in specific situations (such as factory automation), despite decades of research, robotics cannot solve human arm and hand manipulation tasks in everyday unstructured environments. However, in many cases autonomy may not be the goal. Humans naturally interact and collaborate on tasks. A human can easily guide another human how to perform complex tasks. Much of this interaction is non-verbal. For robots to work with humans they need to take visual cues from the human, and be easy to interact with, rather than working autonomously. Hence, with good HRI, humans could relatively effortlessly supervise tasks that are otherwise too complex for robots. ******We propose an integrated systems and experiment-driven approach to this research. In particular we will research:******1. Task specification. We will develop the so called ""task-function"" to handle a combination of camera image-, 3D world- and force sensor coordinates.***2. Video tracking of image features. We will research how to improve robustness of registration-based tracking.***3. Hybrid Visual Servoing. We will extend image-coordinate visual servoing to also work with 3D and force sensory feedback, while requiring a minimum of a-priori calibration.***4. HRI. We will research gesture based human-robot interaction, including what geometric specifications (2D image, 3D, force contacts) are most natural for a human in order to specify complete, provable 3D alignments; and then to sequence into full manipulation tasks. We will develop benchmarks for everyday human manipulation tasks, and study our task specification interfaces on these (e.g. task completion times and cognitive load.)******The goal of our research will be a better understanding of visual and sensory-space motion control, both from a robotics and human interface perspective. Practical outcomes include integration of different tracking and sensory-motor control methods into modular ROS (Robot Operating System) software libraries, and test results of this software by humans on many different manipulation tasks.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2016-06365
Name nom
Jagersand, Martin
Department département
Computing Science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2016
Award amount
54000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Installment
inst:4 - 5
Partie
4
Nb partie
5
Application title
Robot arm and hand manipulation using vision-guided motion control directed by human gestures
Keyword
NSERC
Application summary
This research will focus on the geometric specification of robot (arm and hand) manipulation tasks in camera (and other sensory) coordinate frames. We propose to develop Human-Robot Interfaces (HRI) based on pointing and gestures. The gestures will be interpreted through computer vision and robot motions will be performed through visual servoing, and force servoing. The main objective is to solve general robot arm and hand manipulation tasks interactively with humans.******Robotics research in both engineering and artificial intelligence has focused on autonomy. While autonomy works well in specific situations (such as factory automation), despite decades of research, robotics cannot solve human arm and hand manipulation tasks in everyday unstructured environments. However, in many cases autonomy may not be the goal. Humans naturally interact and collaborate on tasks. A human can easily guide another human how to perform complex tasks. Much of this interaction is non-verbal. For robots to work with humans they need to take visual cues from the human, and be easy to interact with, rather than working autonomously. Hence, with good HRI, humans could relatively effortlessly supervise tasks that are otherwise too complex for robots. ******We propose an integrated systems and experiment-driven approach to this research. In particular we will research:******1. Task specification. We will develop the so called ""task-function"" to handle a combination of camera image-, 3D world- and force sensor coordinates.***2. Video tracking of image features. We will research how to improve robustness of registration-based tracking.***3. Hybrid Visual Servoing. We will extend image-coordinate visual servoing to also work with 3D and force sensory feedback, while requiring a minimum of a-priori calibration.***4. HRI. We will research gesture based human-robot interaction, including what geometric specifications (2D image, 3D, force contacts) are most natural for a human in order to specify complete, provable 3D alignments; and then to sequence into full manipulation tasks. We will develop benchmarks for everyday human manipulation tasks, and study our task specification interfaces on these (e.g. task completion times and cognitive load.)******The goal of our research will be a better understanding of visual and sensory-space motion control, both from a robotics and human interface perspective. Practical outcomes include integration of different tracking and sensory-motor control methods into modular ROS (Robot Operating System) software libraries, and test results of this software by humans on many different manipulation tasks.