Grant: Discovery Grants Program - Individual
McGill University
$28,000.00
Reinforcement Learning Algorithms Designed to Persist
EF 2024-25
Résumé
The field of reinforcement learning is concerned with understanding how intelligent agents can, from trial and error, learn to make decisions that lead to the best outcomes. In silico, its techniques have been applied to wide range of domains, producing computer programs that surpass the world's human champions at the game of Go (2016), can autonomously navigate balloons in the stratosphere (2020), and can design electronics in a fraction of the time taken by human experts (2021).
To achieve this level of performance, however, these programs require weeks or even months or training. This is because the most effective reinforcement learning methods are designed to learn to solve a given task from scratch. Using present algorithms it is difficult, if not downright impossible to carry over the learnings from one version of the program to the next. This makes it hard, for example, to support a learning system that evolves and learns over a period of years - a common scenario in practical applications, where a research and development team might continue to improve the learning software over time.
The research in this proposal aims to address this shortcoming by studying methods and principles with which previously-acquired experience may be carried across iterations of a learning system. Doing so requires understanding how an agent's immediate experience can be synthesized into a more permanent form called a representation of state, and also how an agent can purposefully act to acquire new information that helps it gains a better understanding of its environment. Fundamental advances in this direction will make it possible to design learning systems that benefit from years, if not decades of experience and can therefore make substantially better decisions.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2022-04035
Name nom
Bellemare, Marc
Department département
School of Computer Science
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2022
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Reinforcement Learning Algorithms Designed to Persist
Keywords
reinforcement learning; deep learning; continual learning; applied machine learning; reinforcement learning benchmarks; persistent reinforcement learning
Application summary
The field of reinforcement learning is concerned with understanding how intelligent agents can, from trial and error, learn to make decisions that lead to the best outcomes. In silico, its techniques have been applied to wide range of domains, producing computer programs that surpass the world's human champions at the game of Go (2016), can autonomously navigate balloons in the stratosphere (2020), and can design electronics in a fraction of the time taken by human experts (2021).
To achieve this level of performance, however, these programs require weeks or even months or training. This is because the most effective reinforcement learning methods are designed to learn to solve a given task from scratch. Using present algorithms it is difficult, if not downright impossible to carry over the learnings from one version of the program to the next. This makes it hard, for example, to support a learning system that evolves and learns over a period of years - a common scenario in practical applications, where a research and development team might continue to improve the learning software over time.
The research in this proposal aims to address this shortcoming by studying methods and principles with which previously-acquired experience may be carried across iterations of a learning system. Doing so requires understanding how an agent's immediate experience can be synthesized into a more permanent form called a representation of state, and also how an agent can purposefully act to acquire new information that helps it gains a better understanding of its environment. Fundamental advances in this direction will make it possible to design learning systems that benefit from years, if not decades of experience and can therefore make substantially better decisions.