Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Alberta

$36,000.00

Optimization-based statistical methods: functional estimation, sparsity, compound decisions, and deep learning

EF 2024-25

Résumé
The proposed research aims at the development and further investigation of statistical methodology based on convex optimization, and, at the same time, its long term objectives include the creation of a mathematical toolbox for analyzing perturbations in the pertinent optimization tasks, in particular with regard to an interplay of finite- and infinite-dimensional duality, an important vehicle both for their statistical theory and computational implementations. The specific optimization problems under scrutiny are generally those of semi--infinite character: while they act in infinite-dimensional spaces, they still possess certain finite dimensional features. Proposed areas of investigation comprise the study of perturbations of convex optimization problems applied in density estimation, a continuation of the applicant work in estimation of s-concave densities via Renyi divergences; the extension of this investigation into other similar tasks, and also theoretical investigation of their possible well-posedness in the Hadamard sense and robustness aspects. A proposed outcome of this line of research are also numerical error bounds for the algorithms used in shape-constrained density estimation. The estimation methods based on Renyi divergences are further proposed to be applied also to alternative shape constraints - like the monotonicity of a resulting estimate, with an objective to reveal some yet unknown aspects of the Grenander estimator, but also in the context of mixture models, to obtain novel empirical Bayes predictors with imposed monotonicity. Another proposed topic concerns further understanding of the algorithms for nonparametric maximum likelihood estimation of the mixing probability in mixture models arising in the empirical Bayes framework; their subsequent multidimensional extensions, of cutting-plane and similar nature, are proposed to be investigated as well. This theme continues into the development of the theory of empirical Bayes methods, in particular the interplay of the estimated and "oracle" Bayes predictions, all in the context of general loss functions; the investigation of certain robustness questions in the latter context is proposed as well. The proposed research involves also the exploration of the connection of mixture models and deep learning, based on the discovery that the certain class of neural networks, convex neural networks, are analogous to mixture models; this topic simultaneously aims at the investigation of possible applications of the ideas from nonparametric maximum likelihood estimation to neural networks, and, conversely, that of certain algorithmic techniques from neural networks to the nonparametric estimation of probabilities in mixture models. The last proposed area concerns the development of structure hunting and inference strategies for technologies involving penalized regression methods promoting sparsity, with applications in the frequency analysis of time series and change-point problems.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2020-04424
Name nom
Mizera, Ivan
Department département
Mathematical and Statistical Sciences - Mathematical and Statistical Sciences
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
36000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
3010
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Nonparametric inference
Research subject fr
Inférence non paramétrique
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Optimization-based statistical methods: functional estimation, sparsity, compound decisions, and deep learning
Keywords
statistical methodology; convex and nonconvex optimization; statistical computing; deep learning; compound decisions with mixture models; empirical bayes; functional and density estimation; convex neural networks; quantile regression; sparsity
Application summary
The proposed research aims at the development and further investigation of statistical methodology based on convex optimization, and, at the same time, its long term objectives include the creation of a mathematical toolbox for analyzing perturbations in the pertinent optimization tasks, in particular with regard to an interplay of finite- and infinite-dimensional duality, an important vehicle both for their statistical theory and computational implementations. The specific optimization problems under scrutiny are generally those of semi--infinite character: while they act in infinite-dimensional spaces, they still possess certain finite dimensional features. Proposed areas of investigation comprise the study of perturbations of convex optimization problems applied in density estimation, a continuation of the applicant work in estimation of s-concave densities via Renyi divergences; the extension of this investigation into other similar tasks, and also theoretical investigation of their possible well-posedness in the Hadamard sense and robustness aspects. A proposed outcome of this line of research are also numerical error bounds for the algorithms used in shape-constrained density estimation. The estimation methods based on Renyi divergences are further proposed to be applied also to alternative shape constraints - like the monotonicity of a resulting estimate, with an objective to reveal some yet unknown aspects of the Grenander estimator, but also in the context of mixture models, to obtain novel empirical Bayes predictors with imposed monotonicity. Another proposed topic concerns further understanding of the algorithms for nonparametric maximum likelihood estimation of the mixing probability in mixture models arising in the empirical Bayes framework; their subsequent multidimensional extensions, of cutting-plane and similar nature, are proposed to be investigated as well. This theme continues into the development of the theory of empirical Bayes methods, in particular the interplay of the estimated and "oracle" Bayes predictions, all in the context of general loss functions; the investigation of certain robustness questions in the latter context is proposed as well. The proposed research involves also the exploration of the connection of mixture models and deep learning, based on the discovery that the certain class of neural networks, convex neural networks, are analogous to mixture models; this topic simultaneously aims at the investigation of possible applications of the ideas from nonparametric maximum likelihood estimation to neural networks, and, conversely, that of certain algorithmic techniques from neural networks to the nonparametric estimation of probabilities in mixture models. The last proposed area concerns the development of structure hunting and inference strategies for technologies involving penalized regression methods promoting sparsity, with applications in the frequency analysis of time series and change-point problems.