Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Waterloo

$55,000.00

Robust and Sample Efficient Reinforcement Learning

FY 2023-24

Summary
Reinforcement Learning (RL) is arguably one of the most comprehensive forms of machine learning. It facilitates active learning and it allows a system to learn over an extended period of time about its environment as it makes a sequence of decisions. The system can also learn from weak signals that might be delayed. This is particularly useful in robotics, autonomous vehicles, conversational agents, game playing, operations research, automated trading, non-myopic recommender systems and self-managing networks. The generality of reinforcement learning also makes it complex and therefore challenging algorithmically. Objectives: The goal of this work is to develop algorithms to improve the robustness and sample efficiency of reinforcement learning. Tremendous progress has been achieved in recent years by deep reinforcement learning techniques that scale to high dimensional inputs (e.g., images and natural language) and complex tasks. However, most of the successes are limited to applications with simulated environments (e.g., games, simulated robotic environments) since current algorithms may execute costly/catastrophic actions and may require an amount of data that is prohibitively large for interaction with real environments. Methods: I will develop novel Bayesian reinforcement learning techniques that can quantify the uncertainty of the environment. This will be helpful both for robustness and sample efficiency. In Bayesian learning, a distribution over the unknowns is estimated and refined at each time step. This distribution also allows a system to explore more efficiently by focusing its actions on the parts of the environment that are still unknown. To that effect, I will develop scalable Bayesian techniques for deep reinforcement learning that explore safely and efficiently. I will also develop novel constrained reinforcement learning techniques that take into account secondary objectives such as variance and cost functions that should not be exceeded. This will further improve robustness by ensuring that key performance indicators (KPIs) are met in industrial applications. I will also develop generative reinforcement learning techniques that are robust to missing inputs. In some applications (e.g., non-myopic recommender systems and self-managing networks), some observations/sensors might not be available at each time step. Generative reinforcement learning techniques that can marginalize inputs in a principled way will be designed.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2019-05014
Name nom
Poupart, Pascal
Department département
Computer Science
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
55000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Robust and Sample Efficient Reinforcement Learning
Keyword
Artificial Intelligence; Bayesian Learning; Constrained Optimization; Deep Neural Networks; Machine Learning; Markov Decision Processes; Reinforcement Learning; Sum Product Networks
Application summary
Reinforcement Learning (RL) is arguably one of the most comprehensive forms of machine learning. It facilitates active learning and it allows a system to learn over an extended period of time about its environment as it makes a sequence of decisions. The system can also learn from weak signals that might be delayed. This is particularly useful in robotics, autonomous vehicles, conversational agents, game playing, operations research, automated trading, non-myopic recommender systems and self-managing networks. The generality of reinforcement learning also makes it complex and therefore challenging algorithmically. Objectives: The goal of this work is to develop algorithms to improve the robustness and sample efficiency of reinforcement learning. Tremendous progress has been achieved in recent years by deep reinforcement learning techniques that scale to high dimensional inputs (e.g., images and natural language) and complex tasks. However, most of the successes are limited to applications with simulated environments (e.g., games, simulated robotic environments) since current algorithms may execute costly/catastrophic actions and may require an amount of data that is prohibitively large for interaction with real environments. Methods: I will develop novel Bayesian reinforcement learning techniques that can quantify the uncertainty of the environment. This will be helpful both for robustness and sample efficiency. In Bayesian learning, a distribution over the unknowns is estimated and refined at each time step. This distribution also allows a system to explore more efficiently by focusing its actions on the parts of the environment that are still unknown. To that effect, I will develop scalable Bayesian techniques for deep reinforcement learning that explore safely and efficiently. I will also develop novel constrained reinforcement learning techniques that take into account secondary objectives such as variance and cost functions that should not be exceeded. This will further improve robustness by ensuring that key performance indicators (KPIs) are met in industrial applications. I will also develop generative reinforcement learning techniques that are robust to missing inputs. In some applications (e.g., non-myopic recommender systems and self-managing networks), some observations/sensors might not be available at each time step. Generative reinforcement learning techniques that can marginalize inputs in a principled way will be designed.