Grant: Discovery Grants Program - Individual
Concordia University
$24,000.00
Towards Continual and Compositional Learning in the Visual World
FY 2024-25
Summary
Artificial intelligence and machine learning hold promise to be a major aspect of the future Canadian and global economy. However the current class of machine learning systems still have many fundamental challenges. Existing machine learning and particularly deep learning systems are commonly applied to isolated tasks or narrow domains (such as image recognition of predefined objects), using large amounts of data from the isolated domain. It is also typically assumed that all the samples of the tasks of interest are available during training. However, a realistic scenario and one closer to human-like learning is learning from a stream of data from changing domains or tasks, potentially with little data in some tasks. The objective of such a continual learner is to quickly adapt to new situations or tasks by exploiting previously acquired knowledge, while protecting previous learning from being erased. Critically, this must be accomplished under computation and memory constraints. Such a scenario commonly arises for example in applications ranging from robotic systems to social media platforms. This continual learning problem has recently been studied, with limited success as compared to static systems and a disconnect from real-world applications. In particular many of the recent developments focus on drastic memory limitations, while real life applications are far more restricted in compute budget and have learners observing much longer sequence lengths.
The goals of the proposed research on continual learning will be to develop novel datasets and evaluation protocols relevant for real-world use cases as well as methods and models which can scale to these situations. Although critical in many domains deep learning models have unique properties in terms of computation, memory requirements, and in terms of their online learning abilities. This work will thus focus on deep learning models and their applications. In our initial work we will aim to evaluate a wide variety of existing continual learning approaches on the set of new proposed evaluation metrics and datasets. Then we will propose and evaluate novel methodological directions. We will study the role of memory and experience replay for solving continual learning and devise concurrently low computational complexity ways to utilize prior memories while receiving new data. Additionally we will explore novel deep learning architectural elements in particular based on the conditional computation framework, which can allow the building of reusable components while avoiding learning in irrelevant system modules.
Multiple application areas will be considered with a focus on computer vision and natural language processing. A long term aim will be to connect this work with problems involving computer simulations of robotic agents receiving instructions in a visually rich environment. One use case of such models being to drive household robotics applications.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2021-04104
Name nom
Belilovsky, Eugene
Department département
Department of Computer Science and Software Engineering
Organization id
70
Institution établissement
Concordia University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2021
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Towards Continual and Compositional Learning in the Visual World
Keywords
continual learning; lifelong learning; scene understanding
Application summary
Artificial intelligence and machine learning hold promise to be a major aspect of the future Canadian and global economy. However the current class of machine learning systems still have many fundamental challenges. Existing machine learning and particularly deep learning systems are commonly applied to isolated tasks or narrow domains (such as image recognition of predefined objects), using large amounts of data from the isolated domain. It is also typically assumed that all the samples of the tasks of interest are available during training. However, a realistic scenario and one closer to human-like learning is learning from a stream of data from changing domains or tasks, potentially with little data in some tasks. The objective of such a continual learner is to quickly adapt to new situations or tasks by exploiting previously acquired knowledge, while protecting previous learning from being erased. Critically, this must be accomplished under computation and memory constraints. Such a scenario commonly arises for example in applications ranging from robotic systems to social media platforms. This continual learning problem has recently been studied, with limited success as compared to static systems and a disconnect from real-world applications. In particular many of the recent developments focus on drastic memory limitations, while real life applications are far more restricted in compute budget and have learners observing much longer sequence lengths.
The goals of the proposed research on continual learning will be to develop novel datasets and evaluation protocols relevant for real-world use cases as well as methods and models which can scale to these situations. Although critical in many domains deep learning models have unique properties in terms of computation, memory requirements, and in terms of their online learning abilities. This work will thus focus on deep learning models and their applications. In our initial work we will aim to evaluate a wide variety of existing continual learning approaches on the set of new proposed evaluation metrics and datasets. Then we will propose and evaluate novel methodological directions. We will study the role of memory and experience replay for solving continual learning and devise concurrently low computational complexity ways to utilize prior memories while receiving new data. Additionally we will explore novel deep learning architectural elements in particular based on the conditional computation framework, which can allow the building of reusable components while avoiding learning in irrelevant system modules.
Multiple application areas will be considered with a focus on computer vision and natural language processing. A long term aim will be to connect this work with problems involving computer simulations of robotic agents receiving instructions in a visually rich environment. One use case of such models being to drive household robotics applications.