Grant: Discovery Grants Program - Individual

Intelligent assistive devices such as electrical prosthetic hands can improve the lives of millions of people who unfortunately lost their upper limbs or have limited mobility. However, the majority of such devices do not provide manipulation capabilities, limiting their functionality. Also, the rejection rates for using electrical prostheses are 23% in adults and 32% in children. It is crucial to develop the next generation of robotic and prosthetic hands with sufficient manipulation capabilities to reduce the rejection rates and thus improve the quality of life. One of the significant problems of the current robotic assistants and upper-limb prostheses is a limitation regarding sensor feedback and control for the reliable manipulation of objects in complex and dynamic environments, such as homes, hospitals, schools, etc. The main challenge for robotic manipulation tasks in such settings is that they need accurate tactile sensing to reduce errors caused by miscalculated geometry, friction, forces, camera occlusion, and clutter. Because of that, current robotic hand systems often lack solutions for essential tasks such as object recognition, sensing of physical properties, and after-gasp object pose estimation. Also, current robotic hands usually provide limited tactile feedback of irregular surfaces and objects. Improving tactile sensing for irregular shapes and surfaces would reduce errors and enhance control of such devices, which is crucial to make such hands more reliable in cluttered or occluded situations. Therefore, this research program focuses on enhancing robotic manipulation using tactile sensing and AI, specifically improving after-grasp object pose estimation, object recognition, and physical properties extraction. We will develop new sensors and methods to incorporate efficient and reliable control into assistive devices and prostheses. To achieve these objectives, we will first study tactile sensing data collected from different robotic tasks of robotic and human-like hands. Second, we will investigate how non-normal forces and sensor displacement could integrate with visual data. Finally, we will train machine- and deep-learning models using novel visual and tactile sensing for robotic and human-like hands. A cohort of HQP, including undergraduate, master's, and PhD students, will be trained with this grant over the next five years, improving Canada's research capacity on robotics. Improving the reliability of robotic and human-like hands allows intelligent assistants to perform more tasks and increases the adoption of smart prostheses. Also, making robotic and human-like hands more reliable has significant societal and economic impacts on Canadians.