Grant: Canada Research Chairs
University of Toronto
$25,000.00
Probabilistic Inference and Deep Learning
FY 2023-24
Summary
Neural networks have become the core machine learning technology across many areas of Artificial Intelligence, due to their ability to automatically learn high-level feature representations. While networks have achieved impressive performance in terms of the accuracy of their predictions, there remain several obstacles: neural networks are often overconfident about their predictions, sensitive to shifts in the data distribution, or prone to exploit spurious correlations. Furthermore, neural net architectures are largely based on pattern recognition, whereas extending them to more difficult problems will require more deliberative reasoning.
Much of Dr. Grosse's research has focused on understanding neural net training dynamics – the
ways in which what happens during training affects the final outcome. His program is unique in that while most such work has focused on the pattern recognition setting, he will extend such analyses to cases where the network performs sophisticated reasoning or optimization at test time by meeting the following objectives:
1) extending prior investigations of neural net training dynamics to settings where the architectures perform more sophisticated planning, reasoning, or optimization, and where the training regime may involve multiple neural nets trained to different objectives
2) understanding how and why these settings can lead to different patterns of generalization compared with more traditional neural nets
3) understanding the reasons for a neural net’s predictions by determining how the predictions would have changed if the network were trained on slightly different data; and
4) developing algorithms for training a neural net to produce not only an answer, but also an independently checkable justification for that answer.
Ultimately, this program will lead to greater interpretability of the network’s predictions, as well as reducing the reliance on spurious correlations in the data.
Program
Canada Research Chairs
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
CRC-2021-00500
Name nom
Grosse, Roger
Department département
Computer Science - University of Toronto
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
25000
Program id
CRC
Program name en
Canada Research Chairs
Program name fr
Chaires de recherche du Canada
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
9998
Committee name en
Canada Research Chair Selection Committee
Committee name fr
Comité de sélection des Chaires de recherche du Canada
Area of application code
10000
Area of application group en
The socioeconomic objective available
Area of application group fr
L'objectif socioéconomique disponible
Area of application en
The socioeconomic objective available
Area of application fr
L'objectif socioéconomique disponible
Research subject code
10000
Research subject group en
The field of research available
Research subject group fr
The field of research available
Research subject en
Le domaine de recherche disponible
Research subject fr
Le domaine de recherche disponible
Field of research list codes
RDF1020104
Field of research list names en
Machine learning
Field of research list names fr
Apprentissage machine
Socioeconomic objective list codes
RDS10610
Socioeconomic objective list names en
Information systems, technologies and services
Socioeconomic objective list names fr
Systèmes, technologies et services d'information
Application title
Probabilistic Inference and Deep Learning
Keyword
machine learning; deep learning; probabilistic artificial intelligence; neural network optimization; bilevel optimization; neural network training dynamics; AI alignment
Application summary
Neural networks have become the core machine learning technology across many areas of Artificial Intelligence, due to their ability to automatically learn high-level feature representations. While networks have achieved impressive performance in terms of the accuracy of their predictions, there remain several obstacles: neural networks are often overconfident about their predictions, sensitive to shifts in the data distribution, or prone to exploit spurious correlations. Furthermore, neural net architectures are largely based on pattern recognition, whereas extending them to more difficult problems will require more deliberative reasoning.
Much of Dr. Grosse's research has focused on understanding neural net training dynamics – the
ways in which what happens during training affects the final outcome. His program is unique in that while most such work has focused on the pattern recognition setting, he will extend such analyses to cases where the network performs sophisticated reasoning or optimization at test time by meeting the following objectives:
1) extending prior investigations of neural net training dynamics to settings where the architectures perform more sophisticated planning, reasoning, or optimization, and where the training regime may involve multiple neural nets trained to different objectives
2) understanding how and why these settings can lead to different patterns of generalization compared with more traditional neural nets
3) understanding the reasons for a neural net’s predictions by determining how the predictions would have changed if the network were trained on slightly different data; and
4) developing algorithms for training a neural net to produce not only an answer, but also an independently checkable justification for that answer.
Ultimately, this program will lead to greater interpretability of the network’s predictions, as well as reducing the reliance on spurious correlations in the data.