Grant: Discovery Grants Program - Individual
École Polytechnique de Montréal
$24,000.00
Representation Learning for Lifelong Learning
FY 2024-25
Summary
With the success of Deep Learning (DL), the Machine Learning (ML) community has made huge progress in several challenging tasks such as speech recognition, image classification, and machine translation. Most of this success can be attributed to the ability of Deep Neural Networks (DNNs) to learn meaningful semantic representations that makes the tasks easier to learn. Even though DL methods achieve human-level performance in several tasks, they require millions of examples to learn a task as opposed to humans who require very few examples. This is mainly because we train DNNs to learn a task from scratch while humans exploit their prior knowledge that they acquired from previous tasks. In this research program, I propose to explore the idea of lifelong learning where a single agent can learn a stream of tasks over its lifetime. When compared to agents that learn only one task, a lifelong learning agent can transfer knowledge from one task to another and hence can be sample-efficient in learning new tasks.
Even though lifelong learning has been discussed by the ML community over the last three decades, single-task algorithms have become mature only recently that we can finally make practical progress towards lifelong learning. This research proposal is based on the observation that DNN representations, which are the state-of-the-art for single-task settings, are not suitable for a multi-task lifelong learning setting. This can be justified based on the following two reasons. First, DNNs can forget previous tasks while training for a new task and hence the performance in previous tasks might drop. This is called catastrophic forgetting. Second, the DNN, which is a finite capacity parametric model, can run out of capacity which might force the network to unlearn some previous tasks to learn a new task. We call this the capacity saturation issue. This research program aims to design new representation learning algorithms and novel neural network architectures that are suitable for lifelong learning. I intend to focus on the following four research directions to achieve this goal.
1. Designing modular neural architectures for lifelong learning. Different modules focusing on different functionalities should help in avoiding interference and modular architecture should be easy to expand by adding new modules when compared to monolithic architectures.
2. Designing memory augmented neural networks (MANNs) for lifelong learning. MANNs should help in learning long-term dependencies that arise in lifelong learning.
3. Develop solutions for lifelong reinforcement learning (RL) by using the solutions developed for lifelong supervised learning in the first two research directions.
4. Create strong benchmarks for both lifelong supervised learning and lifelong RL to accelerate the progress in research. Lack of strong benchmarks is an issue in these areas.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-07198
Name nom
Anbil Parthipan, Sarath Chandar
Department département
Département de génie informatique et génie logiciel
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Representation Learning for Lifelong Learning
Keywords
machine learning; deep learning; neural networks; lifelong learning; continual learning; reinforcement learning; memory augmented neural networks; multi-task learning; transfer learning
Application summary
With the success of Deep Learning (DL), the Machine Learning (ML) community has made huge progress in several challenging tasks such as speech recognition, image classification, and machine translation. Most of this success can be attributed to the ability of Deep Neural Networks (DNNs) to learn meaningful semantic representations that makes the tasks easier to learn. Even though DL methods achieve human-level performance in several tasks, they require millions of examples to learn a task as opposed to humans who require very few examples. This is mainly because we train DNNs to learn a task from scratch while humans exploit their prior knowledge that they acquired from previous tasks. In this research program, I propose to explore the idea of lifelong learning where a single agent can learn a stream of tasks over its lifetime. When compared to agents that learn only one task, a lifelong learning agent can transfer knowledge from one task to another and hence can be sample-efficient in learning new tasks.
Even though lifelong learning has been discussed by the ML community over the last three decades, single-task algorithms have become mature only recently that we can finally make practical progress towards lifelong learning. This research proposal is based on the observation that DNN representations, which are the state-of-the-art for single-task settings, are not suitable for a multi-task lifelong learning setting. This can be justified based on the following two reasons. First, DNNs can forget previous tasks while training for a new task and hence the performance in previous tasks might drop. This is called catastrophic forgetting. Second, the DNN, which is a finite capacity parametric model, can run out of capacity which might force the network to unlearn some previous tasks to learn a new task. We call this the capacity saturation issue. This research program aims to design new representation learning algorithms and novel neural network architectures that are suitable for lifelong learning. I intend to focus on the following four research directions to achieve this goal.
1. Designing modular neural architectures for lifelong learning. Different modules focusing on different functionalities should help in avoiding interference and modular architecture should be easy to expand by adding new modules when compared to monolithic architectures.
2. Designing memory augmented neural networks (MANNs) for lifelong learning. MANNs should help in learning long-term dependencies that arise in lifelong learning.
3. Develop solutions for lifelong reinforcement learning (RL) by using the solutions developed for lifelong supervised learning in the first two research directions.
4. Create strong benchmarks for both lifelong supervised learning and lifelong RL to accelerate the progress in research. Lack of strong benchmarks is an issue in these areas.