Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Canada Research Chairs

University of Toronto

$100,000.00

Generative Models

EF 2023-24

Résumé
Deep learning's success in language processing, machine vision, robotics, and predictive models means that many practitioners are eager to apply them to real-world problems. However, those in medicine, business, and the physical sciences still struggle to fit large predictive models in their domains. This is partly because most data in these domains are heterogeneous, messy time series, which mixed structured measurements with unstructured text annotations, usually come at irregular internals, and have complex cross-indexing. Examples include medical histories, user interaction traces, and climatological datasets. My research centres on building new model classes and fitting procedures that are flexible and scalable enough to ``meet the data where it lives'', with little to no pre-processing. End-to-end learning all the way to raw data not only increases the scope of gains in predictive performance, it reduces the organizational and personnel burden of using data effectively. This allows both human and machine judgement to be augmented by more informed predictions. I am developing a family of stochastic partial differential equation models that allow multi-scale approximations of physical systems. Modern physical simulators, such as climate or astrophysics simulations, operate at multiple resolutions: fine-grained levels incorporate detailed sensor data, while coarse-grained representations afford long-range predictions. These simulators are designed largely by hand today, requiring thousands of engineer-years to heuristically program and adjust assimilation mechanisms, and guess which approximations will make the best tradeoffs in speed vs accuracy. Fitting these models requires optimizing millions of parameters in a hierarchy of approximations. The key new ability that my approach has is that both the parameters of the model and its approximations can be jointly learned using scalable gradient-based optimization, without ever having to run expensive simulations using the finest-grained physical laws. This new approach can almost exactly integrate away most latent variables in those regions and times where no observations are made. In other words, this approach will allow for reasoning about observations made far apart in time and space, without requiring a detailed simulation of all the irrelevant details in between. This should allow ultra-scalable simulators for both physical systems and for training huge models from scratch.
Programme
Canada Research Chairs
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
CRC-2021-00226
Name nom
Duvenaud, David
Department département
Computer Science - University of Toronto
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
100000
Program id
CRC
Program name en
Canada Research Chairs
Program name fr
Chaires de recherche du Canada
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
9998
Committee name en
Canada Research Chair Selection Committee
Committee name fr
Comité de sélection des Chaires de recherche du Canada
Area of application code
10000
Area of application group en
The socioeconomic objective available
Area of application group fr
L'objectif socioéconomique disponible
Area of application en
The socioeconomic objective available
Area of application fr
L'objectif socioéconomique disponible
Research subject code
10000
Research subject group en
The field of research available
Research subject group fr
The field of research available
Research subject en
Le domaine de recherche disponible
Research subject fr
Le domaine de recherche disponible
Field of research list codes
RDF1020104
Field of research list names en
Machine learning
Field of research list names fr
Apprentissage machine
Socioeconomic objective list codes
RDS10610
Socioeconomic objective list names en
Information systems, technologies and services
Socioeconomic objective list names fr
Systèmes, technologies et services d'information
Application title
Generative Models
Keyword
machine learning; generative models; deep learning; neural networks; approximate inference; differential equations
Application summary
Deep learning's success in language processing, machine vision, robotics, and predictive models means that many practitioners are eager to apply them to real-world problems. However, those in medicine, business, and the physical sciences still struggle to fit large predictive models in their domains. This is partly because most data in these domains are heterogeneous, messy time series, which mixed structured measurements with unstructured text annotations, usually come at irregular internals, and have complex cross-indexing. Examples include medical histories, user interaction traces, and climatological datasets. My research centres on building new model classes and fitting procedures that are flexible and scalable enough to ``meet the data where it lives'', with little to no pre-processing. End-to-end learning all the way to raw data not only increases the scope of gains in predictive performance, it reduces the organizational and personnel burden of using data effectively. This allows both human and machine judgement to be augmented by more informed predictions. I am developing a family of stochastic partial differential equation models that allow multi-scale approximations of physical systems. Modern physical simulators, such as climate or astrophysics simulations, operate at multiple resolutions: fine-grained levels incorporate detailed sensor data, while coarse-grained representations afford long-range predictions. These simulators are designed largely by hand today, requiring thousands of engineer-years to heuristically program and adjust assimilation mechanisms, and guess which approximations will make the best tradeoffs in speed vs accuracy. Fitting these models requires optimizing millions of parameters in a hierarchy of approximations. The key new ability that my approach has is that both the parameters of the model and its approximations can be jointly learned using scalable gradient-based optimization, without ever having to run expensive simulations using the finest-grained physical laws. This new approach can almost exactly integrate away most latent variables in those regions and times where no observations are made. In other words, this approach will allow for reasoning about observations made far apart in time and space, without requiring a detailed simulation of all the irrelevant details in between. This should allow ultra-scalable simulators for both physical systems and for training huge models from scratch.