Grant: Discovery Grants Program - Individual
University of Waterloo
$27,000.00
Statistical Methods for Data Subject to Label Shift
EF 2024-25
Résumé
Machine learning and statistical models are primarily confined to addressing tasks from the same distribution that generated the data. For example, the training and testing data in classic supervised learning problems have the same distribution. Under such a constraint, however, machine learning and statistical methods are inadequate for addressing real-world problems where distributional shifts usually happen. Distributional shift not only happens in machine learning areas like computer vision and speech recognition but also in biomedical and social science studies. In the healthcare data collection process, for example, some cohorts may be oversampled (e.g., people with better access to healthcare providers), but the interest is on the general public. Thus, failing to address distributional shifts will likely result in fairness and equity concerns. The proposed research program mainly focuses on a specific distributional shift called the label shift. The label shift is complicated and understudied. The long-term goal of the proposed research is to address critical challenges caused by the label shift in areas including artificial intelligence, biomedical studies, and social studies. Current research on the label shift, done mainly by computer scientists, primarily focuses on adjusting point prediction (e.g., classification), while topics like point and interval estimations as well as prediction set (or interval) are understudied. Thus, in the short term, the proposed research aims to borrow the strength of modern semiparametric efficiency theory to achieve the goals of (1) deriving efficient and robust estimators under label shift; (2) improving on existing point prediction adjusting procedures; (3) deriving prediction set/interval using conformal prediction; (4) extending the label shift assumption to widen its applicability to more complicated settings. The proposed research will train HQPs in modern statistical tools and machine learning techniques, including state-of-the-art deep learning methods. In addition to theories, the HQPs will learn computing skills by applying the newly developed methodologies to real datasets and simulations. Thus, preparing the HQPs for their future careers in academia or industry. The research outputs will help advance statistical research and benefit Canadian interdisciplinary research, due to the prevalence of label shift problems in areas like artificial intelligence, biomedical studies, econometrics, and social science.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2023-03479
Name nom
Tian, Qinglong
Department département
Department of Statistics and Actuarial Science
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
27000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1211
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Medical and health sciences
Area of application fr
Sciences médicales et de la santé
Research subject code
3001
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistical theory
Research subject fr
Théorie de statistique
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Statistical Methods for Data Subject to Label Shift
Keywords
label shift; semiparametric efficiency; classification; uncertainty quantification; conformal prediction; transfer learning; distributional shift; efficient estimator
Application summary
Machine learning and statistical models are primarily confined to addressing tasks from the same distribution that generated the data. For example, the training and testing data in classic supervised learning problems have the same distribution. Under such a constraint, however, machine learning and statistical methods are inadequate for addressing real-world problems where distributional shifts usually happen. Distributional shift not only happens in machine learning areas like computer vision and speech recognition but also in biomedical and social science studies. In the healthcare data collection process, for example, some cohorts may be oversampled (e.g., people with better access to healthcare providers), but the interest is on the general public. Thus, failing to address distributional shifts will likely result in fairness and equity concerns. The proposed research program mainly focuses on a specific distributional shift called the label shift. The label shift is complicated and understudied. The long-term goal of the proposed research is to address critical challenges caused by the label shift in areas including artificial intelligence, biomedical studies, and social studies. Current research on the label shift, done mainly by computer scientists, primarily focuses on adjusting point prediction (e.g., classification), while topics like point and interval estimations as well as prediction set (or interval) are understudied. Thus, in the short term, the proposed research aims to borrow the strength of modern semiparametric efficiency theory to achieve the goals of (1) deriving efficient and robust estimators under label shift; (2) improving on existing point prediction adjusting procedures; (3) deriving prediction set/interval using conformal prediction; (4) extending the label shift assumption to widen its applicability to more complicated settings. The proposed research will train HQPs in modern statistical tools and machine learning techniques, including state-of-the-art deep learning methods. In addition to theories, the HQPs will learn computing skills by applying the newly developed methodologies to real datasets and simulations. Thus, preparing the HQPs for their future careers in academia or industry. The research outputs will help advance statistical research and benefit Canadian interdisciplinary research, due to the prevalence of label shift problems in areas like artificial intelligence, biomedical studies, econometrics, and social science.