Grant: Discovery Grants Program - Individual
University of Northern British Columbia
$17,000.00
Significance measure and result replicability in prediction modelling
FY 2011-12
Summary
Result replicability is considered fundamental in scientific research methodology. Statistical significance does not evaluate result replicability. The p-value as a measure of significance dominates statistical analysis and decision making however it surprisingly tells little about replicability. It is not uncommon to observe that in one simulation of small number of repeats of a typical experiment, p-value can vary over a large interval. Thus illustrating that p-value is a very unreliable measure. Both external and internal replicability analyses are important. True external replication studies are many times impractical, and therefore researchers turn to internal methods like cross-validation, the jackknife and the bootstrap. Recently copula functions have emerged as an appropriate measure of dependence among random variables and provide an alternative model to the commonly practised correlation based models for studying the dependence.
This research will help researchers and practitioners both to understand problems associated with the p-value as a measure of statistical significance and problems in using correlation based prediction modelling. Further, it will discuss appropriate alternative significance and dependence measures and models as a remedy. By analysing climatic behaviour of two principal observables, temperature and precipitation, I anticipate that my research will provide methodology leading to better climatic variable prediction models.
Program
Discovery Grants Program - Individual
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
240005-2009
Name nom
Kumar, Pranesh
Department département
Mathematics
Organization id
3
Institution établissement
University of Northern British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2011
Competition year année de concours
2009
Award amount
17000
Program id
RGPIN
Program nane en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
14
Committee name en
Statistical Sciences
Committee name fr
Statistique
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
3000
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistics and probability
Research subject fr
Statistique et probabilité
Partie
3
Nb partie
5
Application title
Significance measure and result replicability in prediction modelling
Keyword
NSERC
Application summary
Result replicability is considered fundamental in scientific research methodology. Statistical significance does not evaluate result replicability. The p-value as a measure of significance dominates statistical analysis and decision making however it surprisingly tells little about replicability. It is not uncommon to observe that in one simulation of small number of repeats of a typical experiment, p-value can vary over a large interval. Thus illustrating that p-value is a very unreliable measure. Both external and internal replicability analyses are important. True external replication studies are many times impractical, and therefore researchers turn to internal methods like cross-validation, the jackknife and the bootstrap. Recently copula functions have emerged as an appropriate measure of dependence among random variables and provide an alternative model to the commonly practised correlation based models for studying the dependence.
This research will help researchers and practitioners both to understand problems associated with the p-value as a measure of statistical significance and problems in using correlation based prediction modelling. Further, it will discuss appropriate alternative significance and dependence measures and models as a remedy. By analysing climatic behaviour of two principal observables, temperature and precipitation, I anticipate that my research will provide methodology leading to better climatic variable prediction models.