Grant: Discovery Grants Program - Individual
Université de Montréal
$50,000.00
Bayesian deep-learning prediction with sparse graphs
EF 2023-24
Résumé
Many statistical problems involving images or relational data are described by relational processes. Among these are inference problems in functional magnetic resonance images (fMRI), social networks, and gene expression data. Also, recently, kernel-based sparse graphs have been introduced to explicitly model the partition data structure in Bayesian non-parametric (BNP) models, that otherwise was only induced as a side-effect of BNP models. Depending on the problem at hand, the likelihood, or the prior density of relational data may be modeled by sparse graph models. Some popular models are the Ising model, and the Potts models, in the discrete case; and Gibbs random fields, in the general case. The use of complex, realistic graph models to account for data relationships have the potential to dramatically improve the prediction of simple models such as multivariate linear regression. On the other hand, complex models such as deep-learning neural networks have proven to be able to predict well when no simpler model is known to work for very complex functional or high dimensional data, provided that the training data is chosen in a suitable manner. Usually large amounts of data are needed to fit the networks. We believe that good prediction can be achieve with less data if the network or other likelihood model is guided by auxiliary models that only use covariate information. Following the success of the semi-parametric Potts regression model, we propose to guide shallow and deep-learning by combining these type of complex predictors with a random partition model based on covariate proximity. The resulting model in the case of a neural network model is a semi-parametric Bayesian mixture neural network whose posterior predictions may be seen as averages over a large collection of possible partitions of the data. We aim at the development of methodologies for sparse-graphs in deep-learning, and of BNP models with explicit random partition structure driven by kernel-based sparse-graphs. The fitting and posterior prediction of the models may be done with Markov chain Monte Carlo methods. Though for the sake of efficiency, we proposed to realize approximate, but accurate, inference through variational methods, graph theory, and techniques derived from graph models in physics. Applications of particular interest in this project are inference for voxel activity in brain fMR images, and networks for prediction in social science.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-05444
Name nom
Murua, Alejandro
Department département
mathématiques et de statistique
Organization id
63
Institution établissement
Université de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
50000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2804
Research subject group en
Artificial intelligence
Research subject group fr
Expert systems
Research subject en
Intelligence artificielle
Research subject fr
Systèmes experts
Application title
Bayesian deep-learning prediction with sparse graphs
Keyword
Bayesian non-parametrics; classification; computational statistics; deep-learning; Gibbs random fields; neural networks; Potts model; prediction; sparse graphs; statistical learning
Application summary
Many statistical problems involving images or relational data are described by relational processes. Among these are inference problems in functional magnetic resonance images (fMRI), social networks, and gene expression data. Also, recently, kernel-based sparse graphs have been introduced to explicitly model the partition data structure in Bayesian non-parametric (BNP) models, that otherwise was only induced as a side-effect of BNP models. Depending on the problem at hand, the likelihood, or the prior density of relational data may be modeled by sparse graph models. Some popular models are the Ising model, and the Potts models, in the discrete case; and Gibbs random fields, in the general case. The use of complex, realistic graph models to account for data relationships have the potential to dramatically improve the prediction of simple models such as multivariate linear regression. On the other hand, complex models such as deep-learning neural networks have proven to be able to predict well when no simpler model is known to work for very complex functional or high dimensional data, provided that the training data is chosen in a suitable manner. Usually large amounts of data are needed to fit the networks. We believe that good prediction can be achieve with less data if the network or other likelihood model is guided by auxiliary models that only use covariate information. Following the success of the semi-parametric Potts regression model, we propose to guide shallow and deep-learning by combining these type of complex predictors with a random partition model based on covariate proximity. The resulting model in the case of a neural network model is a semi-parametric Bayesian mixture neural network whose posterior predictions may be seen as averages over a large collection of possible partitions of the data. We aim at the development of methodologies for sparse-graphs in deep-learning, and of BNP models with explicit random partition structure driven by kernel-based sparse-graphs. The fitting and posterior prediction of the models may be done with Markov chain Monte Carlo methods. Though for the sake of efficiency, we proposed to realize approximate, but accurate, inference through variational methods, graph theory, and techniques derived from graph models in physics. Applications of particular interest in this project are inference for voxel activity in brain fMR images, and networks for prediction in social science.