Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Toronto

$42,000.00

Using a data-centric approach to improve the generalizability of AI models for medical applications

EF 2024-25

Résumé
Over the last 8 years deep learning has dominated the field of medical image analysis and have been applied to a wide variety of imaging modalities to solve classification, segmentation, and prediction problems. Although this has led to the development of some very accurate models in the experimental setting, relatively few AI models have been deployed in the clinic. A key barrier to adoption is that it is difficult to make models robust and generalizable. Health care settings are extremely variable and differences in equipment, scanning protocols and even patient populations can degrade the performance of models trained on one domain and deployed on another. One way of overcoming this problem is to fine-tune existing models trained in one domain to improve their accuracy in the target domain but this requires labelled training data which may be difficult and expensive to obtain, particularly if adaptation has to occur at every site. Another approach is to train models using methods designed to reduce domain dependence, for example aggressive augmentation of training data or adversarial domain adaptation, but with these methods it is still necessary to have labelled data from the target domain to assess accuracy and fine-tuning may still be necessary. We have demonstrated that the use of networks pretrained using self-supervision can significantly reduce the amount of data need to fine-tune models in histopathology. However, we do not have a systematic way of determining what data is suitable for pre-training; for example, with some applications it is better to use a diverse training dataset representing many different tissue types and stains, whilst for other tasks it may be desirable to pre-train using a dataset as similar to the target domain as possible. The size or capacity of the pre-trained network is also something that is usually determined by trial and error: a large network may capture more of the variability in the source dataset, but a smaller model may be more robust to domain shift in the target data. We also need methods or heuristics for determining a priori how much labelled data is needed to achieve the required accuracy in the fine-tuned model, and for choosing the most informative data examples to label. We therefore propose to develop new methods to detect and quantify the presence of domain drift. We will also expand this approach to assess whether individual training cases are out-of-distribution and to identify the most useful cases to add to the training dataset during active learning approaches. By shifting from a model-centric approach, where the emphasis is on designing new model architectures, to a data-centric approach where the emphasis is on the quality of the data used to train the model, we hope to improve the robustness and generalizability of AI models bringing them closer to clinical deployment.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2023-05358
Name nom
Martel, Anne
Department département
Medical Biophysics - Medical Biophysics
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
42000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1102
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Biomedical engineering
Area of application fr
Génie biomédical
Research subject code
1900
Research subject group en
Biomedical engineering
Research subject group fr
Génie biomédical
Research subject en
Biomedical engineering
Research subject fr
Génie biomédical
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Using a data-centric approach to improve the generalizability of AI models for medical applications
Keywords
artificial intelligence; machine learning; weakly supervised learning; medical image analysis; representation learning; generalizability
Application summary
Over the last 8 years deep learning has dominated the field of medical image analysis and have been applied to a wide variety of imaging modalities to solve classification, segmentation, and prediction problems. Although this has led to the development of some very accurate models in the experimental setting, relatively few AI models have been deployed in the clinic. A key barrier to adoption is that it is difficult to make models robust and generalizable. Health care settings are extremely variable and differences in equipment, scanning protocols and even patient populations can degrade the performance of models trained on one domain and deployed on another. One way of overcoming this problem is to fine-tune existing models trained in one domain to improve their accuracy in the target domain but this requires labelled training data which may be difficult and expensive to obtain, particularly if adaptation has to occur at every site. Another approach is to train models using methods designed to reduce domain dependence, for example aggressive augmentation of training data or adversarial domain adaptation, but with these methods it is still necessary to have labelled data from the target domain to assess accuracy and fine-tuning may still be necessary. We have demonstrated that the use of networks pretrained using self-supervision can significantly reduce the amount of data need to fine-tune models in histopathology. However, we do not have a systematic way of determining what data is suitable for pre-training; for example, with some applications it is better to use a diverse training dataset representing many different tissue types and stains, whilst for other tasks it may be desirable to pre-train using a dataset as similar to the target domain as possible. The size or capacity of the pre-trained network is also something that is usually determined by trial and error: a large network may capture more of the variability in the source dataset, but a smaller model may be more robust to domain shift in the target data. We also need methods or heuristics for determining a priori how much labelled data is needed to achieve the required accuracy in the fine-tuned model, and for choosing the most informative data examples to label. We therefore propose to develop new methods to detect and quantify the presence of domain drift. We will also expand this approach to assess whether individual training cases are out-of-distribution and to identify the most useful cases to add to the training dataset during active learning approaches. By shifting from a model-centric approach, where the emphasis is on designing new model architectures, to a data-centric approach where the emphasis is on the quality of the data used to train the model, we hope to improve the robustness and generalizability of AI models bringing them closer to clinical deployment.