Grant: Discovery Grants Program - Individual
University of Alberta
$62,000.00
Learning Predictive Representations from Incomplete Data
EF 2013-14
Résumé
Almost all data is now digitally stored, with unprecedented computing power available to process it. These developments have created new opportunities to advance computer interpretation (e.g. natural language processing, computer perception) and intelligent data-analysis (e.g. bio-informatics) by analyzing the massive amounts of complex data stored in text, multimedia, and scientific repositories. My research addresses the fundamental challenge of synthesizing predictive models from data, focusing on the problem of learning predictors in the presence of latent variables and incomplete observations. These challenges are heightened by the fact that predictions in complex domains are not just simple class labels or scalar values, but are complex structured outputs---such as parse trees, scene labellings or graph labellings---that involve multiple outputs to be predicted in a coordinated fashion, usually with intervening latent variables. The key problem is training complex predictors when some of the output or intervening latent variables are unobserved. To tackle these problems, I will develop convex training principles that combine model optimization with inference of missing components. Convexity decouples parameter optimization from model quality: a poor result arises from poor modeling choices, not a poor local minimum---thus separating specification from implementation. A key insight is that training with incomplete data can be tackled by treating missing components as auxiliary variables to be optimized (i.e. inferred) simultaneously with parameter optimization. Convex formulations of joint training and inference can then be obtained by one of two strategies that I have been developing: working with relaxed equivalence relations over missing components, or deriving implicitly induced regularizers. These approaches have already led to fundamental advances in unsupervised and semi-supervised training, including state of the art methods for dimensionality reduction, robust estimation, and latent large margin models. One of my long term goals is to commoditize solution methods for challeng- ing machine learning formulations, such as predictive representation learning and data component discovery.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
217337-2013
Name nom
Schuurmans, Dale
Department département
Computing Science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2013
Competition year année de concours
2013
Award amount
62000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
1
Nb partie
5
Application title
Learning Predictive Representations from Incomplete Data
Keyword
NSERC
Application summary
Almost all data is now digitally stored, with unprecedented computing power available to process it. These developments have created new opportunities to advance computer interpretation (e.g. natural language processing, computer perception) and intelligent data-analysis (e.g. bio-informatics) by analyzing the massive amounts of complex data stored in text, multimedia, and scientific repositories. My research addresses the fundamental challenge of synthesizing predictive models from data, focusing on the problem of learning predictors in the presence of latent variables and incomplete observations. These challenges are heightened by the fact that predictions in complex domains are not just simple class labels or scalar values, but are complex structured outputs---such as parse trees, scene labellings or graph labellings---that involve multiple outputs to be predicted in a coordinated fashion, usually with intervening latent variables. The key problem is training complex predictors when some of the output or intervening latent variables are unobserved. To tackle these problems, I will develop convex training principles that combine model optimization with inference of missing components. Convexity decouples parameter optimization from model quality: a poor result arises from poor modeling choices, not a poor local minimum---thus separating specification from implementation. A key insight is that training with incomplete data can be tackled by treating missing components as auxiliary variables to be optimized (i.e. inferred) simultaneously with parameter optimization. Convex formulations of joint training and inference can then be obtained by one of two strategies that I have been developing: working with relaxed equivalence relations over missing components, or deriving implicitly induced regularizers. These approaches have already led to fundamental advances in unsupervised and semi-supervised training, including state of the art methods for dimensionality reduction, robust estimation, and latent large margin models. One of my long term goals is to commoditize solution methods for challeng- ing machine learning formulations, such as predictive representation learning and data component discovery.