Grant: Discovery Grants Program - Individual
University of Victoria
$39,000.00
Tighter error bounds for representation learning and lifelong learning
FY 2022-23
Summary
The success of machine learning algorithms crucially hinges on which numerical features are used to represent data. An arguably large part of the success of human learning is that we not only learn representations to perform well on prediction tasks, but we also reuse these representations to more efficiently learn newly encountered but similar tasks. We are now in an era where "deep" machine learning methods actually learn useful representations of data automatically. These deep learning methods are becoming vital to many other fields: just a few successful applications include drug design, 3D rendering for computer graphics, and beating top-level human players in games such as Go. Despite these great empirical successes, our mathematical understanding for why these methods work well and how to best train them is lacking. Part I of my research is to develop new theory to understand how well a deep model will perform when it makes predictions about new data. My strategy to obtain better performance guarantees is to use theory that leverages specific properties of the actual data an algorithm sees and specific properties of the algorithm itself. By considering both of these important aspects of training, I expect to achieve success for deep learning models as well. Another important part of my analysis is to answer the following question: If a deep learning model is insensitive to certain types of transformations of its input, can it successfully be trained using less data as a result?One of the greatest advancements the field of machine learning can make is to shift from learning each new thing in isolation to reusing what has been learned in the past when learning new tasks. This continual transfer when learning an endless sequence of tasks is known as lifelong learning. While some research has begun in this important area, the mathematical theory for how well algorithms can perform lifelong learning is lacking, especially with regards to adaptive algorithms that transfer much more from their past experiences when they encounter tasks that are highly similar to past tasks. Part II of my research is to design algorithms for lifelong learning, including developing mathematical guarantees for these algorithms. I aim to answer questions like the following: If a learning agent encounters a series of slowly changing learning tasks, can we pool the data from previous tasks in order to learn new tasks using much less data? My strategy will be to adapt powerful ideas from sequential prediction, a vital yet previously untapped resource for work in lifelong learning.Both parts of this work can fundamentally advance the field of machine learning, which already is revolutionizing a number of sciences and industries. The students participating in this research will be excellently-equipped to fuel Canada's technological contributions at the global stage.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2018-03942
Name nom
Mehta, Nishant
Department département
Computer Science - Computer Science
Organization id
7
Institution établissement
University of Victoria
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2018
Award amount
39000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Installment
inst:5
Partie
1
Nb partie
6
Application title
Tighter error bounds for representation learning and lifelong learning
Application summary
The success of machine learning algorithms crucially hinges on which numerical features are used to represent data. An arguably large part of the success of human learning is that we not only learn representations to perform well on prediction tasks, but we also reuse these representations to more efficiently learn newly encountered but similar tasks. We are now in an era where "deep" machine learning methods actually learn useful representations of data automatically. These deep learning methods are becoming vital to many other fields: just a few successful applications include drug design, 3D rendering for computer graphics, and beating top-level human players in games such as Go. Despite these great empirical successes, our mathematical understanding for why these methods work well and how to best train them is lacking. Part I of my research is to develop new theory to understand how well a deep model will perform when it makes predictions about new data. My strategy to obtain better performance guarantees is to use theory that leverages specific properties of the actual data an algorithm sees and specific properties of the algorithm itself. By considering both of these important aspects of training, I expect to achieve success for deep learning models as well. Another important part of my analysis is to answer the following question: If a deep learning model is insensitive to certain types of transformations of its input, can it successfully be trained using less data as a result?One of the greatest advancements the field of machine learning can make is to shift from learning each new thing in isolation to reusing what has been learned in the past when learning new tasks. This continual transfer when learning an endless sequence of tasks is known as lifelong learning. While some research has begun in this important area, the mathematical theory for how well algorithms can perform lifelong learning is lacking, especially with regards to adaptive algorithms that transfer much more from their past experiences when they encounter tasks that are highly similar to past tasks. Part II of my research is to design algorithms for lifelong learning, including developing mathematical guarantees for these algorithms. I aim to answer questions like the following: If a learning agent encounters a series of slowly changing learning tasks, can we pool the data from previous tasks in order to learn new tasks using much less data? My strategy will be to adapt powerful ideas from sequential prediction, a vital yet previously untapped resource for work in lifelong learning.Both parts of this work can fundamentally advance the field of machine learning, which already is revolutionizing a number of sciences and industries. The students participating in this research will be excellently-equipped to fuel Canada's technological contributions at the global stage.