Grant: Alliance Grants
University of Toronto
$47,400.00
Reinforcement learning framework of Sim2Real for Robotic applications
FY 2023-24
Summary
One of the primary goals of artificial intelligence (AI) is to create intelligent machines capable of performing tasks in the world by themselves, without explicit human control. Examples range from autonomous driving and helicopters and autonomous robots in collaborative manufacturing and precision livestock farming. Reinforcement learning (RL), one of the sub-categories of AI aiming to solve sequential decision-making problems through trial and error, has been increasingly adopted by the robotics community over the past years to control complex robotic tasks. Despite its remarkable advancement, RL faces important challenges, especially when applied to real-world robotic applications where acquiring sufficient interaction samples can be prohibitively expensive.In the proposed project, we leverage simulation-based training in which autonomous agent(s) is trained with the virtual model of the physical system in a simulator to avoid wear and tear of the expensive system. Simulation-based training provides data at a low cost but involves inherent mismatches with real-world settings. Accordingly, our proposed approach addresses two control problems: (1) minimization of the mismatch between the simulator and physical system, and (2) maximization of the performance of the target control task. Our proposed framework is needed to improve the effectiveness and safety of non-invasive medical procedures typically performed by humans, such as vaccine administration, ultrasound, and needle biopsies, through consistent delivery from reinforcement learning and being able to better adapt to variability. The Sim2Real framework, as a part of the Cobi platform, will be implemented across several hospitals and clinics within Canada to help alleviate labor shortages and empower health care professionals to allocate their time and energy to more impactful and complex tasks.
Program
Alliance Grants
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
576932-2022
Name nom
Lee, Chi-Guhn C-G
Department département
Mechanical and Industrial Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
47400
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
1206
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Engineering
Area of application fr
Génie
Research subject code
2601
Research subject group en
Robotics
Research subject group fr
Robotic control
Research subject en
Robotique
Research subject fr
Systèmes de commande
Application title
Reinforcement learning framework of Sim2Real for Robotic applications
Keyword
Artificial Intelligence; Control Systems; Machine Learning; Non-invasive Medical Procedures; Reinforcement Learning; Robotics; Robotics Simulation; Sim-to-real Transfer; Transfer Learning
Application summary
One of the primary goals of artificial intelligence (AI) is to create intelligent machines capable of performing tasks in the world by themselves, without explicit human control. Examples range from autonomous driving and helicopters and autonomous robots in collaborative manufacturing and precision livestock farming. Reinforcement learning (RL), one of the sub-categories of AI aiming to solve sequential decision-making problems through trial and error, has been increasingly adopted by the robotics community over the past years to control complex robotic tasks. Despite its remarkable advancement, RL faces important challenges, especially when applied to real-world robotic applications where acquiring sufficient interaction samples can be prohibitively expensive.In the proposed project, we leverage simulation-based training in which autonomous agent(s) is trained with the virtual model of the physical system in a simulator to avoid wear and tear of the expensive system. Simulation-based training provides data at a low cost but involves inherent mismatches with real-world settings. Accordingly, our proposed approach addresses two control problems: (1) minimization of the mismatch between the simulator and physical system, and (2) maximization of the performance of the target control task. Our proposed framework is needed to improve the effectiveness and safety of non-invasive medical procedures typically performed by humans, such as vaccine administration, ultrasound, and needle biopsies, through consistent delivery from reinforcement learning and being able to better adapt to variability. The Sim2Real framework, as a part of the Cobi platform, will be implemented across several hospitals and clinics within Canada to help alleviate labor shortages and empower health care professionals to allocate their time and energy to more impactful and complex tasks.