Grant: Discovery Grants Program - Individual
University of Toronto
$28,000.00
Statistical Evidence, Bias and Prior-data Conflict
EF 2024-25
Résumé
The basic idea underlying this research is that any statistical methodology should be based on a clear definition of how statistical evidence, and the strength of this evidence, is to be measured. The particular approach taken is called relative belief based on the principle of evidence which, in its simplest form, says that if the probability of a hypothesis, whose truth value is unknown, increases from its initial assignment after observing data, then the data is providing evidence that the hypothesis is true and if the probability decreases, then the data is providing evidence that the hypothesis is false. Furthermore, the strength of this evidence is assessed by how strongly it is believed what the evidence is saying as measured by a posterior probability. This basic idea leads to a theory of hypothesis assessment and to a theory of estimation and prediction.
To enable application of these ideas, a statistician must specify various ingredients such as a sampling model and a prior. A significant component of this research is concerned with how these should be chosen and in particular how a prior should be elicited. Since these ingredients are chosen rather than being prescribed by a particular application, they are subjective in nature. As such, these ingredients must be subjected to tests of falsification involving the objective part of a statistical investigation, namely, the data. So, another aspect of the research is model checking and checking for prior-data conflict to ensure that the ingredients are not strongly distorting what the evidence in the data is saying.
It is possible to choose the ingredients for a statistical analysis to bias the resulting inferences. Assessing and controlling this through bias calculations is another aspect of this research. Bias against a hypothesis H is measured by the prior probability of getting evidence against H when H is true and bias in favor of H is measured by the prior probability of getting evidence in favor of H when H is false. There are also such bias calculations for the estimation problem where these are concerned with prior coverage probabilities (confidence and accuracy) for the set of values for which there is evidence in favor. Both of these biases can be controlled by sample size. The bias calculations establish an important link between Bayesian inference and frequentism.
There are a number of projects in the proposal that are concerned with extending and applying these ideas to significant statistical problems. For example, the elicitation of priors is a challenging problem as it is necessary to not only provide a methodology for doing this but to also provide computational methodology for implementing the subsequent analysis when the prior takes on an inconvenient form. One key overall goal is to provide easy to use software so that practitioners can use the relative belief approach without having to write extensive amounts of code.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2024-03839
Name nom
Evans, Michael
Department département
Computer and Mathematical Sciences
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
3000
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistics and probability
Research subject fr
Statistique et probabilité
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Statistical Evidence, Bias and Prior-data Conflict
Keywords
foundations of statistical reasoning; eliciting priors; measuring and controlling biases; model checking; checking for prior data conflict; measuring evidence and its strength; relative belief inferences; statistical computation
Application summary
The basic idea underlying this research is that any statistical methodology should be based on a clear definition of how statistical evidence, and the strength of this evidence, is to be measured. The particular approach taken is called relative belief based on the principle of evidence which, in its simplest form, says that if the probability of a hypothesis, whose truth value is unknown, increases from its initial assignment after observing data, then the data is providing evidence that the hypothesis is true and if the probability decreases, then the data is providing evidence that the hypothesis is false. Furthermore, the strength of this evidence is assessed by how strongly it is believed what the evidence is saying as measured by a posterior probability. This basic idea leads to a theory of hypothesis assessment and to a theory of estimation and prediction.
To enable application of these ideas, a statistician must specify various ingredients such as a sampling model and a prior. A significant component of this research is concerned with how these should be chosen and in particular how a prior should be elicited. Since these ingredients are chosen rather than being prescribed by a particular application, they are subjective in nature. As such, these ingredients must be subjected to tests of falsification involving the objective part of a statistical investigation, namely, the data. So, another aspect of the research is model checking and checking for prior-data conflict to ensure that the ingredients are not strongly distorting what the evidence in the data is saying.
It is possible to choose the ingredients for a statistical analysis to bias the resulting inferences. Assessing and controlling this through bias calculations is another aspect of this research. Bias against a hypothesis H is measured by the prior probability of getting evidence against H when H is true and bias in favor of H is measured by the prior probability of getting evidence in favor of H when H is false. There are also such bias calculations for the estimation problem where these are concerned with prior coverage probabilities (confidence and accuracy) for the set of values for which there is evidence in favor. Both of these biases can be controlled by sample size. The bias calculations establish an important link between Bayesian inference and frequentism.
There are a number of projects in the proposal that are concerned with extending and applying these ideas to significant statistical problems. For example, the elicitation of priors is a challenging problem as it is necessary to not only provide a methodology for doing this but to also provide computational methodology for implementing the subsequent analysis when the prior takes on an inconvenient form. One key overall goal is to provide easy to use software so that practitioners can use the relative belief approach without having to write extensive amounts of code.