Grant: Discovery Grants Program - Individual
Université de Montréal
$19,000.00
Kernel-based non-parametric Bayesian clustering models
FY 2016-17
Summary
This proposal is based on the belief that the key to the success of modeling complex data structures lies in the embedding of kernel-based or kernel-induced methods within Bayesian non-parametric models. The Bayesian paradigm gives a rich framework for modeling structural data constraints, whereas kernel-based methods incorporates flexibility (e.g. local, spatial and/or temporal adaptation). Two main ingredients proposed are the introduction of locally adaptive kernels, and the use of discrete Gibbs random fields (such as the auto-logistic and Potts models) and determinantal point processes as priors or penalization for the unknown clustering data structure in classification and regression models. We can show that the adaptive kernels are linked to kernel density and regression estimators. Consequently, the theory developed for these estimators can be adapted to more complex learning algorithms. Discrete Gibbs random fields offer an interesting alternative to the commonly used Dirichlet process priors. Their clusters are linked to data graphs. Therefore, the rich theory developed for discrete graphs may be used to guide the choice of parameters and their estimation. Determinantal point processes offer another alternative to guide the discovery of clusters.Computational techniques such as the Wang-Landau algorithm will be extended to our models in an efficient way. For example, in some cases, multi-dimensional parameter dependent quantities may be factorized in lower-dimensional parameter quantities so as to reduced the complexity of the stochastic sampling algorithms. Bayesian variational techniques, such as the mean-field approximation, will be developed explicitly to address the computational needs of our models. Indeed, for the particular case of the Potts model, several other better approximating variational techniques have been suggested in the Physics literature. We will study, adapt and extend these techniques to more complex models for clustering for classification and regression.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
327689-2013
Name nom
Murua, Alejandro
Department département
Mathématiques et de statistique
Organization id
63
Institution établissement
Université de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2016
Competition year année de concours
2013
Award amount
19000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1102
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Biomedical engineering
Area of application fr
Génie biomédical
Research subject code
2804
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Expert systems
Research subject fr
Systèmes experts
Partie
4
Nb partie
5
Application title
Kernel-based non-parametric Bayesian clustering models
Keyword
NSERC
Application summary
This proposal is based on the belief that the key to the success of modeling complex data structures lies in the embedding of kernel-based or kernel-induced methods within Bayesian non-parametric models. The Bayesian paradigm gives a rich framework for modeling structural data constraints, whereas kernel-based methods incorporates flexibility (e.g. local, spatial and/or temporal adaptation). Two main ingredients proposed are the introduction of locally adaptive kernels, and the use of discrete Gibbs random fields (such as the auto-logistic and Potts models) and determinantal point processes as priors or penalization for the unknown clustering data structure in classification and regression models. We can show that the adaptive kernels are linked to kernel density and regression estimators. Consequently, the theory developed for these estimators can be adapted to more complex learning algorithms. Discrete Gibbs random fields offer an interesting alternative to the commonly used Dirichlet process priors. Their clusters are linked to data graphs. Therefore, the rich theory developed for discrete graphs may be used to guide the choice of parameters and their estimation. Determinantal point processes offer another alternative to guide the discovery of clusters.Computational techniques such as the Wang-Landau algorithm will be extended to our models in an efficient way. For example, in some cases, multi-dimensional parameter dependent quantities may be factorized in lower-dimensional parameter quantities so as to reduced the complexity of the stochastic sampling algorithms. Bayesian variational techniques, such as the mean-field approximation, will be developed explicitly to address the computational needs of our models. Indeed, for the particular case of the Potts model, several other better approximating variational techniques have been suggested in the Physics literature. We will study, adapt and extend these techniques to more complex models for clustering for classification and regression.