Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Alliance Grants

University of Waterloo

$25,000.00

Realtime Reinforcement Learning for Mechatronic Systems with Non-Stationary Behaviors

FY 2023-24

Summary
Conventional mechatronic systems such as industrial robots operated mainly for repetitive and dull tasks in isolation from human workers. In contrast, modern mechatronic systems (e.g., driverless cars, robot vacuums, collaborative robots, etc.) are getting increasingly more complex and agile, and are required to operate under more uncertain environments. The increased complexity and uncertainties associated with these systems made conventional control strategies (that are based on physical models) not sufficient. As an alternative, a data-driven decision-making strategy called reinforcement learning (RL) has been promoted in recent years. However, RL still has critical limitations for real-life physical systems. Two major issues are 1) that RL requires a large amount of test data for training, and 2) that it cannot effectively handle time-varying or non-stationary behaviours such as variations in payload, mechanical wear and tear, communication delays, etc. This proposal tackles the second issue through international collaboration between two experts in automatic control from University of Waterloo and POSTECH (Pohang University of Science and Technology) in Korea. Specifically, we aim to develop a new algorithmic framework for RL that can i) detect variations in environmental conditions through constant monitoring of its performance, ii) systematically vary RL policy toward sufficient exploration, and iii) build reconfigurable policy to choose optimal action according to varying environmental conditions. Additional effort will be made to advance specialized knowledge in implementing a new RL framework for real-life physical systems through collaborative coding and joint experimental testing. The outcome of this project is expected to accelerate the application of AI and machine learning to real-life physical systems to make them smarter and more efficient. We envision that the proposed research will help Canada remain at the forefront of these disruptive technologies, and eventually lead to innovating industrial processes and plants, improving quality of life, and creating more high-quality jobs.
Program
Alliance Grants
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
586324-2023
Name nom
Jeon, Soo S
Department département
Mechanical Engineering and Mechatronics
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2023
Award amount
25000
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
500
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Manufacturing processes and products
Area of application fr
Procédés et produits industriels
Research subject code
2601
Research subject group en
Robotics
Research subject group fr
Robotic control
Research subject en
Robotique
Research subject fr
Systèmes de commande
Application title
Realtime Reinforcement Learning for Mechatronic Systems with Non-Stationary Behaviors
Keyword
autonomous robots; data-oriented approaches; mechatronics; model-based techniques; motion control; reinforcement learning
Application summary
Conventional mechatronic systems such as industrial robots operated mainly for repetitive and dull tasks in isolation from human workers. In contrast, modern mechatronic systems (e.g., driverless cars, robot vacuums, collaborative robots, etc.) are getting increasingly more complex and agile, and are required to operate under more uncertain environments. The increased complexity and uncertainties associated with these systems made conventional control strategies (that are based on physical models) not sufficient. As an alternative, a data-driven decision-making strategy called reinforcement learning (RL) has been promoted in recent years. However, RL still has critical limitations for real-life physical systems. Two major issues are 1) that RL requires a large amount of test data for training, and 2) that it cannot effectively handle time-varying or non-stationary behaviours such as variations in payload, mechanical wear and tear, communication delays, etc. This proposal tackles the second issue through international collaboration between two experts in automatic control from University of Waterloo and POSTECH (Pohang University of Science and Technology) in Korea. Specifically, we aim to develop a new algorithmic framework for RL that can i) detect variations in environmental conditions through constant monitoring of its performance, ii) systematically vary RL policy toward sufficient exploration, and iii) build reconfigurable policy to choose optimal action according to varying environmental conditions. Additional effort will be made to advance specialized knowledge in implementing a new RL framework for real-life physical systems through collaborative coding and joint experimental testing. The outcome of this project is expected to accelerate the application of AI and machine learning to real-life physical systems to make them smarter and more efficient. We envision that the proposed research will help Canada remain at the forefront of these disruptive technologies, and eventually lead to innovating industrial processes and plants, improving quality of life, and creating more high-quality jobs.