Grant: Discovery Grants Program - Individual
Simon Fraser University
$31,000.00
Meta-learning and uncertainty quantification for statistical learning
EF 2024-25
Résumé
Statistical learning algorithms (SLAs) are popular data analysis tools that remain a mystery to many novice users. They can be used to relate subjects' response measure to other features measured on the subject and to produce predicted responses from a given set of features. They have an advantage over traditional statistical models in their ability to adapt to any relationships apparent from the data, rather than imposing assumptions for mathematical and interpretative convenience. But they also have disadvantages: (1) most SLAs simply create predictions for a given set of inputs without a measure of certainty or confidence in those predictions, and (2) they can be difficult to use without extensive trial-and-error computational work.
The long-term goal of my research program is to develop data analysis tools that overcome these shortcomings. We will provide the tools to users in the form of functions and packages in R, a popular statistical analysis platform, and as apps accessible on any computer.
To address the problem of quantifying uncertainty, we will focus on three statistical prediction issues: predicting the time until a particular event occurs, predicting the times of multiple events, and predicting whether multiple events will occur. In each case, we will develop SLA versions that produce not only predicted values, but also measures of the uncertainty associated with these predictions.
To make SLAs more user-friendly, we focus on the common problem of regression analysis, relating a numeric response measurement to a set of features on each subject. Many SLAs exist for this problem, but their performance can vary drastically among data sets, likely due to inherent properties of the data sets, like size of the data set and the strength and complexity of relationships within it. We will develop measures of many of these properties as well as comparative performance of SLAs on these data sets. From this, we will build a model to predict which SLAs might work best on any given data set.
Making SLAs more accessible and applicable to varied problems is hugely important for improving the analysis of data sets used by researchers and practitioners across Canada and worldwide. We will create the first-ever SLAs with valid measures of uncertainty for several important data-analysis problems. Further, because using a poor statistical analysis technique can give answers that are highly flawed, this research will guide analysts to select the appropriate SLA that will perform well on their particular data sets.
My research program will provide trainees with opportunities to gain skills and experience in cutting-edge data analysis and methods development. These skills are in high demand, as ever more sophisticated methods of statistical analysis will need to be developed to match the increasing complexity of data collected.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2024-05146
Name nom
Loughin, Thomas
Department département
Statistics and Actuarial Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
31000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
3000
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistics and probability
Research subject fr
Statistique et probabilité
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Meta-learning and uncertainty quantification for statistical learning
Keywords
meta-learning; meta-features; random forest; regression trees; machine learning; prediction intervals; credible intervals; bart
Application summary
Statistical learning algorithms (SLAs) are popular data analysis tools that remain a mystery to many novice users. They can be used to relate subjects' response measure to other features measured on the subject and to produce predicted responses from a given set of features. They have an advantage over traditional statistical models in their ability to adapt to any relationships apparent from the data, rather than imposing assumptions for mathematical and interpretative convenience. But they also have disadvantages: (1) most SLAs simply create predictions for a given set of inputs without a measure of certainty or confidence in those predictions, and (2) they can be difficult to use without extensive trial-and-error computational work.
The long-term goal of my research program is to develop data analysis tools that overcome these shortcomings. We will provide the tools to users in the form of functions and packages in R, a popular statistical analysis platform, and as apps accessible on any computer.
To address the problem of quantifying uncertainty, we will focus on three statistical prediction issues: predicting the time until a particular event occurs, predicting the times of multiple events, and predicting whether multiple events will occur. In each case, we will develop SLA versions that produce not only predicted values, but also measures of the uncertainty associated with these predictions.
To make SLAs more user-friendly, we focus on the common problem of regression analysis, relating a numeric response measurement to a set of features on each subject. Many SLAs exist for this problem, but their performance can vary drastically among data sets, likely due to inherent properties of the data sets, like size of the data set and the strength and complexity of relationships within it. We will develop measures of many of these properties as well as comparative performance of SLAs on these data sets. From this, we will build a model to predict which SLAs might work best on any given data set.
Making SLAs more accessible and applicable to varied problems is hugely important for improving the analysis of data sets used by researchers and practitioners across Canada and worldwide. We will create the first-ever SLAs with valid measures of uncertainty for several important data-analysis problems. Further, because using a poor statistical analysis technique can give answers that are highly flawed, this research will guide analysts to select the appropriate SLA that will perform well on their particular data sets.
My research program will provide trainees with opportunities to gain skills and experience in cutting-edge data analysis and methods development. These skills are in high demand, as ever more sophisticated methods of statistical analysis will need to be developed to match the increasing complexity of data collected.