Grant: Discovery Grants Program - Individual
University of Alberta
$48,000.00
Leveraging Human and Agent Guidance for Improved Reinforcement Learning
EF 2023-24
Résumé
Reinforcement learning (RL) is a type of machine learning that lets virtual or physical agents learn through experience, often finding novel solutions to difficult problems and exceeding human performance. RL has had many exciting successes from video game playing to data center optimization. Unfortunately, there are still relatively few real-world, deployed, RL success stories. One reason is that learning a policy can be very slow and there is an emphasis on agents learning from scratch. In contrast, this research will better allow RL agents to learn from others. This research will enable more deployments of RL in real-world scenarios by using existing knowledge from humans and agents to jumpstart initial behavior and reach high performing policies more quickly. An RL agent student can receive help from a human or agent teacher with multiple kinds of guidance, such as demonstration, action advice, or direct reward feedback. When successful, this research will enable RL to be successfully deployed in more real-world scenarios by focusing costly exploration and jumpstarting initial behavior to quickly reach high quality policies. The goal of this student/teacher framework is to improve the student's learning (relative to learning without guidance) without harming the agent's final performance. An additional goal can be to have the student outperform the teacher. The research is divided into three specific aims. Aim 1 focuses on how agents can best use human guidance, when different types of guidance are more or less useful, and how humans want to provide guidance. Aim 2 considers when a student should ask for guidance, or when a teacher should proactively provide guidance. Aim 3 considers the more general case when a student can learn from multiple teachers and when multiple students can learn from a single teacher. A key criticism of RL is that it can be slow to learn and that initial performance can be poor. By leveraging other agents, programs, human experts, and human non-experts as teachers, this research will help create opportunities across industries where RL successfully learns in physical and virtual settings to impact people. Not only will this research program help create Canadian jobs by using RL to improve processes in existing companies, it may help RL create new opportunities for businesses and startups that do not currently exist. Graduate students involved in this research will develop critical research, machine learning, and human-AI interaction skills. Other research groups will benefit from developed software, as it will enable standardization and make human subject studies in RL more accessible.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2021-02538
Name nom
Taylor, Matthew
Department département
computing science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
48000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Leveraging Human and Agent Guidance for Improved Reinforcement Learning
Keyword
Human-AI interaction; Inter-agent teaching; Learning from Humans; Multi-agent systems; Reinforcement Learning
Application summary
Reinforcement learning (RL) is a type of machine learning that lets virtual or physical agents learn through experience, often finding novel solutions to difficult problems and exceeding human performance. RL has had many exciting successes from video game playing to data center optimization. Unfortunately, there are still relatively few real-world, deployed, RL success stories. One reason is that learning a policy can be very slow and there is an emphasis on agents learning from scratch. In contrast, this research will better allow RL agents to learn from others. This research will enable more deployments of RL in real-world scenarios by using existing knowledge from humans and agents to jumpstart initial behavior and reach high performing policies more quickly. An RL agent student can receive help from a human or agent teacher with multiple kinds of guidance, such as demonstration, action advice, or direct reward feedback. When successful, this research will enable RL to be successfully deployed in more real-world scenarios by focusing costly exploration and jumpstarting initial behavior to quickly reach high quality policies. The goal of this student/teacher framework is to improve the student's learning (relative to learning without guidance) without harming the agent's final performance. An additional goal can be to have the student outperform the teacher. The research is divided into three specific aims. Aim 1 focuses on how agents can best use human guidance, when different types of guidance are more or less useful, and how humans want to provide guidance. Aim 2 considers when a student should ask for guidance, or when a teacher should proactively provide guidance. Aim 3 considers the more general case when a student can learn from multiple teachers and when multiple students can learn from a single teacher. A key criticism of RL is that it can be slow to learn and that initial performance can be poor. By leveraging other agents, programs, human experts, and human non-experts as teachers, this research will help create opportunities across industries where RL successfully learns in physical and virtual settings to impact people. Not only will this research program help create Canadian jobs by using RL to improve processes in existing companies, it may help RL create new opportunities for businesses and startups that do not currently exist. Graduate students involved in this research will develop critical research, machine learning, and human-AI interaction skills. Other research groups will benefit from developed software, as it will enable standardization and make human subject studies in RL more accessible.