Grant: Discovery Grants Program - Individual
McGill University
$16,000.00
Semi-parametric multidimensional item response models for large-scale and operational testing
EF 2019-20
Résumé
I develop, program, and evaluate innovative latent variable models. This work has broad applications across the social/health sciences, yet the main challenges are in statistics (primarily NSERC topic MS28).******As an example, large-scale/operational testing spans multiple disciplines (education, health, licensure, psychology) and often makes use of item response theory (IRT). IRT provides statistical tools to estimate the properties of items, construct tests, and score individual respondents. Such scores are often used for policy or high-stakes decisions.******In this context, psychometric models must often meet one or more demands: 1) Estimation with multiple groups and planned missing data designs; 2) Fast estimation with a long test and many respondents; 3) Feasibility with a computer adaptive test (CAT); and 4) Interpretability by stakeholders.******Such applications increasingly use multidimensional IRT (MIRT), which allows measurement of multiple substantive constructs. While there are several semiparametric and nonparametric IRT models, there are arguably no MIRT approaches that can both relax typical parametric assumptions and meet the above demands. In fact, many unidimensional semi/nonparametric approaches require more research to improve estimation speed, or use with multiple groups, missing data, and in a CAT.******This gap in modeling capacity is critical to address because use of a restrictive parametric model or ignoring multidimensionality can lead to poor estimates of item properties and individuals' scores. Therefore, the long-term objective of the proposed research program is integrate MIRT and semi/nonparametric modeling to develop new models that can meet the above demands.******Over the next five years, I will extend my work on monotonic polynomial (MP) models to the case of MIRT. I argue that MP-based models have interpretational advantages and can be estimated using maximum marginal likelihood to facilitate multiple groups and missing data. Much additional work is required to achieve the following short-term objectives:***1. The development and estimation of new MP-based MIRT models.***2. Enhancement of the estimation speed of MP-based models through use of metaheuristic optimization.***3. Development and testing of MP-based models for use in computer adaptive testing.******This research requires analytical/technical work (Objectives 1 and 3) and statistical computing (all Objectives). Real data and Monte Carlo simulations will compare new MP-based approaches versus extant nonparametric and parametric (M)IRT models.******Given a lack of alternatives, new semiparametric MIRT models are original and a potentially groundbreaking addition to current MIRT capabilities. Such models can enhance our knowledge of semi/nonparametric approaches and metaheuristics in general, improve the validity of large-scale/operational tests, and will serve as a benchmark for future developments.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2018-05357
Name nom
Falk, Carl
Department département
Psychology
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2018
Award amount
16000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
3004
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Applied statistics
Research subject fr
Statistique appliquée
Installment
NA
Partie
0
Nb partie
NA
Application title
Semi-parametric multidimensional item response models for large-scale and operational testing
Keyword
NSERC
Application summary
I develop, program, and evaluate innovative latent variable models. This work has broad applications across the social/health sciences, yet the main challenges are in statistics (primarily NSERC topic MS28).******As an example, large-scale/operational testing spans multiple disciplines (education, health, licensure, psychology) and often makes use of item response theory (IRT). IRT provides statistical tools to estimate the properties of items, construct tests, and score individual respondents. Such scores are often used for policy or high-stakes decisions.******In this context, psychometric models must often meet one or more demands: 1) Estimation with multiple groups and planned missing data designs; 2) Fast estimation with a long test and many respondents; 3) Feasibility with a computer adaptive test (CAT); and 4) Interpretability by stakeholders.******Such applications increasingly use multidimensional IRT (MIRT), which allows measurement of multiple substantive constructs. While there are several semiparametric and nonparametric IRT models, there are arguably no MIRT approaches that can both relax typical parametric assumptions and meet the above demands. In fact, many unidimensional semi/nonparametric approaches require more research to improve estimation speed, or use with multiple groups, missing data, and in a CAT.******This gap in modeling capacity is critical to address because use of a restrictive parametric model or ignoring multidimensionality can lead to poor estimates of item properties and individuals' scores. Therefore, the long-term objective of the proposed research program is integrate MIRT and semi/nonparametric modeling to develop new models that can meet the above demands.******Over the next five years, I will extend my work on monotonic polynomial (MP) models to the case of MIRT. I argue that MP-based models have interpretational advantages and can be estimated using maximum marginal likelihood to facilitate multiple groups and missing data. Much additional work is required to achieve the following short-term objectives:***1. The development and estimation of new MP-based MIRT models.***2. Enhancement of the estimation speed of MP-based models through use of metaheuristic optimization.***3. Development and testing of MP-based models for use in computer adaptive testing.******This research requires analytical/technical work (Objectives 1 and 3) and statistical computing (all Objectives). Real data and Monte Carlo simulations will compare new MP-based approaches versus extant nonparametric and parametric (M)IRT models.******Given a lack of alternatives, new semiparametric MIRT models are original and a potentially groundbreaking addition to current MIRT capabilities. Such models can enhance our knowledge of semi/nonparametric approaches and metaheuristics in general, improve the validity of large-scale/operational tests, and will serve as a benchmark for future developments.