Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Toronto

$40,000.00

Learning with Discrete Structure

EF 2023-24

Résumé
Deep learning has revolutionized many subfields of artificial intelligence. Deep learning models, originally inspired by the human brain and often called neural nets, can be trained to mimic very complex input-output relationships. Examples of input-output relationships include the written text that corresponds to a given recording of speech or the type of object present in a given image. Deep learning models excel at extracting subtle statistical patterns from their inputs and can be used to produce very accurate predictions of the outputs. Our ability to train deep learning models has improved dramatically over the years, which has revolutionized many applications where accurate predictions are important. Consider OpenAI's GPT3 system. GPT3 is a natural language model that learns to predict the next word in a sentence. Once trained, the model is modified to solve a wide range of tasks, including semantic search, customer service, and content comprehension. Access to the trained model is offered via a simple web interface. The quality of GPT3's predictions is so high that humans have trouble distinguishing them from human predictions, and the system now serves millions of queries a day. Due in part to successes like this, there is a sense that the revolution will eventually come to every domain. Industrial and medical applications are often cited as the next frontier for deep learning. In many of these applications the data is known to have discrete structure. For example, in drug discovery methods, one is often interested in predicting whether a candidate molecule has a certain property. Molecules can be represented as graphs in which nodes correspond to atoms and edges correspond to chemical bonds. This discrete structure can be used to improve the predictions of machine learning systems. For another example, consider the problem of linear optimization with integer constraints. These are challenging combinatorial optimization problems, and some researchers have proposed deep learning models that directly predict optimal solutions. In this setting, the output, which should satisfy the constraints in the problem specification, is a structured discrete object. Despite the promise of deep learning, challenges remain in its application to discrete structured data. Attempts to learn the heuristics of combinatorial solvers, for instance, are typically held back by the slowness of neural net computations. In molecular drug design, deep learning mail fail to produce interpretable models, making it difficult for scientists to take full advantage of deep learning predictions. The long-term goal of my research program is to address these challenges and deliver on the promise of deep learning for discrete structured data.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2021-03445
Name nom
Maddison, Christopher
Department département
Computer Science
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
40000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Learning with Discrete Structure
Keyword
Approximate Bayesian inference; Differentiable convex optimization layers; Generative models; Gradient Estimation; Latent variable models; Machine learning; Monte Carlo methods; Relational inference; Structured learning
Application summary
Deep learning has revolutionized many subfields of artificial intelligence. Deep learning models, originally inspired by the human brain and often called neural nets, can be trained to mimic very complex input-output relationships. Examples of input-output relationships include the written text that corresponds to a given recording of speech or the type of object present in a given image. Deep learning models excel at extracting subtle statistical patterns from their inputs and can be used to produce very accurate predictions of the outputs. Our ability to train deep learning models has improved dramatically over the years, which has revolutionized many applications where accurate predictions are important. Consider OpenAI's GPT3 system. GPT3 is a natural language model that learns to predict the next word in a sentence. Once trained, the model is modified to solve a wide range of tasks, including semantic search, customer service, and content comprehension. Access to the trained model is offered via a simple web interface. The quality of GPT3's predictions is so high that humans have trouble distinguishing them from human predictions, and the system now serves millions of queries a day. Due in part to successes like this, there is a sense that the revolution will eventually come to every domain. Industrial and medical applications are often cited as the next frontier for deep learning. In many of these applications the data is known to have discrete structure. For example, in drug discovery methods, one is often interested in predicting whether a candidate molecule has a certain property. Molecules can be represented as graphs in which nodes correspond to atoms and edges correspond to chemical bonds. This discrete structure can be used to improve the predictions of machine learning systems. For another example, consider the problem of linear optimization with integer constraints. These are challenging combinatorial optimization problems, and some researchers have proposed deep learning models that directly predict optimal solutions. In this setting, the output, which should satisfy the constraints in the problem specification, is a structured discrete object. Despite the promise of deep learning, challenges remain in its application to discrete structured data. Attempts to learn the heuristics of combinatorial solvers, for instance, are typically held back by the slowness of neural net computations. In molecular drug design, deep learning mail fail to produce interpretable models, making it difficult for scientists to take full advantage of deep learning predictions. The long-term goal of my research program is to address these challenges and deliver on the promise of deep learning for discrete structured data.