Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Waterloo

$50,000.00

Safe Reinforcement Learning for Real-World Autonomous Driving in Urban Environments

EF 2024-25

Résumé
Reinforcement learning (RL) can be defined as learning from interactions to achieve a goal. While RL is a promising method for control systems design, its application to real-world safety-critical tasks, such as motion planning and control of autonomous vehicles (AVs), is still challenging. This method involves interactions between an agent and its environment, where a wide range of actions should be explored, including many failed iterations, before achieving the desired goal. When RL is deployed on a physical system like AV, there is a possibility of unsafe actions (decisions) by the agent (AV's controller) that can damage the vehicle or hurt people. As a result, certifying safety is essential during learning and deploying RL for real-life AV applications. While other techniques have been proposed to achieve RL with safety features, action projection methods can provide the closest provably safe action. An elegant action projection approach can be obtained by coupling RL and Control Barrier Functions (CBFs). We can apply RL-CBF techniques to find proper RL actions for AVs, like changing their speeds or steering angles, while satisfying safety constraints, such as keeping safe distances from nearby vehicles. Although RL-CBF methods have great potential, most RL-CBF AV motion planners and controllers proposed previously have been designed with unrealistic assumptions, and limited studies exist for complex urban driving scenarios like unsignalized intersections and roundabouts. This research program will advance existing RL-CBF AV safety-critical controllers by considering more realistic models and constraints while addressing optimization infeasibilities and modeling uncertainties. The resulting advanced RL-CBF AV controllers will be evaluated and compared against one another and with enhanced versions of promising non-learning CBF-based controllers proposed in the past. For our comparative study and to determine the best RL-CBF AV controller for urban driving, we will first run high-fidelity simulations. For a more effective evaluation of the best identified RL-CBF AV controller, we will conduct hardware-in-the-loop tests and real-world AV experiments. As a result, we will develop a novel RL-CBF control synthesis framework for real-world AV safety-critical tasks applicable to highly complex, arbitrary, and unseen driving scenarios in urban environments. Our research will add critical competitive values to AVs built by Canadian car companies while making their products safer for real-life applications. It will also help address the upcoming shortage of skilled engineers for the fast-growing AV job market. The proposed research will offer unique opportunities to combine RL with elegant control-theoretic techniques to ensure safety that will be applicable to other areas like unmanned aerial vehicles and human-robot collaboration. This will contribute to putting Canada at the forefront of safety-critical control systems design innovations.
Programme
Discovery Grants Program - Individual
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2023-03678
Name nom
Lashgarian Azad, Nasser
Department département
Systems Design Engineering
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
50000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1512
Committee name en
Mechanical Engineering
Committee name fr
Génie mécanique
Area of application code
700
Area of application group en
Transportation systems and services
Area of application group fr
Services et systèmes de transport
Area of application en
Transportation systems and services
Area of application fr
Services et systèmes de transport
Research subject code
2108
Research subject group en
Mechanical engineering
Research subject group fr
Génie mécanique
Research subject en
Aerospace, aeronautical and automotive engineering
Research subject fr
Génie aérospatial, aéronautique et automobile
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Safe Reinforcement Learning for Real-World Autonomous Driving in Urban Environments
Keywords
autonomous vehicle motion planning and control; reinforcement learning; control barrier functions; hardware-in-the-loop tests; safety-critical control; model uncertainties; urban driving scenarios; optimization infeasibilities; high-fidelity simulations; full-scale autonomous golf cart experiments
Application summary
Reinforcement learning (RL) can be defined as learning from interactions to achieve a goal. While RL is a promising method for control systems design, its application to real-world safety-critical tasks, such as motion planning and control of autonomous vehicles (AVs), is still challenging. This method involves interactions between an agent and its environment, where a wide range of actions should be explored, including many failed iterations, before achieving the desired goal. When RL is deployed on a physical system like AV, there is a possibility of unsafe actions (decisions) by the agent (AV's controller) that can damage the vehicle or hurt people. As a result, certifying safety is essential during learning and deploying RL for real-life AV applications. While other techniques have been proposed to achieve RL with safety features, action projection methods can provide the closest provably safe action. An elegant action projection approach can be obtained by coupling RL and Control Barrier Functions (CBFs). We can apply RL-CBF techniques to find proper RL actions for AVs, like changing their speeds or steering angles, while satisfying safety constraints, such as keeping safe distances from nearby vehicles. Although RL-CBF methods have great potential, most RL-CBF AV motion planners and controllers proposed previously have been designed with unrealistic assumptions, and limited studies exist for complex urban driving scenarios like unsignalized intersections and roundabouts. This research program will advance existing RL-CBF AV safety-critical controllers by considering more realistic models and constraints while addressing optimization infeasibilities and modeling uncertainties. The resulting advanced RL-CBF AV controllers will be evaluated and compared against one another and with enhanced versions of promising non-learning CBF-based controllers proposed in the past. For our comparative study and to determine the best RL-CBF AV controller for urban driving, we will first run high-fidelity simulations. For a more effective evaluation of the best identified RL-CBF AV controller, we will conduct hardware-in-the-loop tests and real-world AV experiments. As a result, we will develop a novel RL-CBF control synthesis framework for real-world AV safety-critical tasks applicable to highly complex, arbitrary, and unseen driving scenarios in urban environments. Our research will add critical competitive values to AVs built by Canadian car companies while making their products safer for real-life applications. It will also help address the upcoming shortage of skilled engineers for the fast-growing AV job market. The proposed research will offer unique opportunities to combine RL with elegant control-theoretic techniques to ensure safety that will be applicable to other areas like unmanned aerial vehicles and human-robot collaboration. This will contribute to putting Canada at the forefront of safety-critical control systems design innovations.