Grant: Discovery Grants Program - Individual
Simon Fraser University
$43,000.00
Bayes assisted model assessment
FY 2022-23
Summary
The problem of assessing model assumptions has two features of current importance. First, statisticians continue to develop myriad new ad hoc procedures. Second we are making more and more use of unidentifiable models. The resulting difficulties are handled by making uncheckable assumptions; these assumptions are often couched in the language of rates of convergence as the number of parameters, p, grows with some measure of sample size, say n. High dimensional models and inference procedures often showcase both. I am engaged in a long term program to try to reduce the extent to which procedures have only such ad hoc motivations and to understand in practical terms the impact of the non-identifiability. I am particularly focussed on model assessment and validation. I consider statistical models in which there are assumptions about some function. In goodness-of-fit, my main area of work, these assumptions are about distribution functions or densities but the ideas are also relevant to regression models (the regression function) or multivariate analysis (assumptions about a copula) or time series (assumptions about a spectral distribution). The main thrust of my research is then to assess the assumptions. My current goal is to use Bayesian ideas to find and evaluate frequentist tests. Objectives of the proposed research program: we will be developing and assessing hypothesis tests of modelling assumptions. Our framework starts with a finite dimensional parametric specification for some function. The null hypothesis is then either a parametric model or a semi-parametric specification in which it is the parametric specification of the function which is to be checked. The alternative is then specified via an infinite dimensional nonparametric model extending the null model. We put priors on the alternative seeking to capture the nature of reasonable alternatives, then describe and study the resulting optimal tests. The priors are treated as a tool in developing the test, as a tool in comparing tests and, by computing posteriors after rejection of the hypothesized model, as a guide to how to modify the model. We will be trying to use these tools to study well known tests with a view to learning what, if any, priors would lead to such tests. We want tests for distributional assumptions: assumptions about the distribution of the data, or of the errors in a regression model or for latent variables (random effects such as frailties). We will work to develop tests for regression functions, for link functions, for spectral densities, and many more such. More broadly still I hope to use priors to develop and evaluate frequentist procedures for multistep inferential processes. We are also working on applying prior distributions in much smaller problems such as on-off experiments, replicating our work on particle discovery in high energy physics in a much simpler framework in order to combine good frequency theory properties with seriously informative priors.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2021-03185
Name nom
Lockhart, Richard
Department département
Statistics and Actuarial Science - Statistics and Actuarial Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2021
Award amount
43000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
3000
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistics and probability
Research subject fr
Statistique et probabilité
Installment
inst:2
Partie
1
Nb partie
5
Application title
Bayes assisted model assessment
Application summary
The problem of assessing model assumptions has two features of current importance. First, statisticians continue to develop myriad new ad hoc procedures. Second we are making more and more use of unidentifiable models. The resulting difficulties are handled by making uncheckable assumptions; these assumptions are often couched in the language of rates of convergence as the number of parameters, p, grows with some measure of sample size, say n. High dimensional models and inference procedures often showcase both. I am engaged in a long term program to try to reduce the extent to which procedures have only such ad hoc motivations and to understand in practical terms the impact of the non-identifiability. I am particularly focussed on model assessment and validation. I consider statistical models in which there are assumptions about some function. In goodness-of-fit, my main area of work, these assumptions are about distribution functions or densities but the ideas are also relevant to regression models (the regression function) or multivariate analysis (assumptions about a copula) or time series (assumptions about a spectral distribution). The main thrust of my research is then to assess the assumptions. My current goal is to use Bayesian ideas to find and evaluate frequentist tests. Objectives of the proposed research program: we will be developing and assessing hypothesis tests of modelling assumptions. Our framework starts with a finite dimensional parametric specification for some function. The null hypothesis is then either a parametric model or a semi-parametric specification in which it is the parametric specification of the function which is to be checked. The alternative is then specified via an infinite dimensional nonparametric model extending the null model. We put priors on the alternative seeking to capture the nature of reasonable alternatives, then describe and study the resulting optimal tests. The priors are treated as a tool in developing the test, as a tool in comparing tests and, by computing posteriors after rejection of the hypothesized model, as a guide to how to modify the model. We will be trying to use these tools to study well known tests with a view to learning what, if any, priors would lead to such tests. We want tests for distributional assumptions: assumptions about the distribution of the data, or of the errors in a regression model or for latent variables (random effects such as frailties). We will work to develop tests for regression functions, for link functions, for spectral densities, and many more such. More broadly still I hope to use priors to develop and evaluate frequentist procedures for multistep inferential processes. We are also working on applying prior distributions in much smaller problems such as on-off experiments, replicating our work on particle discovery in high energy physics in a much simpler framework in order to combine good frequency theory properties with seriously informative priors.