Grant: Discovery Grants Program - Individual
University of Toronto
$82,000.00
Robust machine learning for healthcare
FY 2024-25
Summary
The great promise of AI in healthcare is taking time to materialize. Besides difficulties with access to the data and unrealistic expectations of the AI due to the hype fueled by the media, there are many fundamental machine learning advances that need to be made to achieve the widespread use of AI in healthcare. Some of the current shortcomings include but are not limited to 1) high error rates in predicting rare critical events (such as cardiac arrest); ii) lack of model robustness to the underlying changes in the data arising due to changes in the policy and practice; iii) lack of the model explainability that makes it hard for the users, e.g. clinicians, to act upon predictions that diverge from their intuition. In my lab, we have started addressing some of these fundamental issues. In this grant I am proposing a research program to make substantial improvements in addressing these questions over the next 5 years.
Classification vs outlier detection The frequency of many critical events, such as cardiac arrest, is usually below 5%. These problems are often incorrectly treated as classification. One of the goals of our work will be to expand the class of generative models, creating models fit for the purpose of outlier detection in healthcare.
Dealing with shifts in the data. Patient populations, care practices, and database systems evolve over time, and yet few works to date have explicitly addressed data shift and evolution as part of the medical record modeling. We showed that there is a simple yet effective mitigation strategy: aggregation of raw features into expert defined clinical concepts. We will develop new robust feature representation techniques and adaptive learning over time to make increase readiness of machine learning (ML) for healthcare deployment in the future.
Explainability. Once developed, translating ML models effectively to practice requires establishing users' trust. We surveyed clinicians to understand their perception of interpretable models. The next step is to integrate these perceptions of explainability into the ML model development. In the program proposed in this grant I will explore counterfactual models to achieve this goal.
Training. My work draws on and affects many disciplines, from computer science to medicine to ethics. Over the last eight years, I have been training HQP coming from a variety of backgrounds, from computer science to biology, from engineering to economics as well as medical professionals in using and developing new AI models for healthcare. It is my goal to supervise and help grow this new and burgeoning field to result in a very highly skilled workforce.
Impact. The amount of available health data has made it clear that machine learning has a great role to play in the future of medicine. The work proposed in this grant will help to build the foundation for responsible AI+healthcare to ensure the realization of the AI potential in this important field.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-05777
Name nom
Goldenberg, Anna
Department département
Computer Science (St. George Campus) - Computer Science (St. George Campus)
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
82000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1101
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Human health (including medically-related psychological research)
Area of application fr
Santé publique (y compris recherche psychologique liée à la médecine)
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Robust machine learning for healthcare
Keywords
machine learning; healthcare; explainable models; counterfactual models; data shift; generative models
Application summary
The great promise of AI in healthcare is taking time to materialize. Besides difficulties with access to the data and unrealistic expectations of the AI due to the hype fueled by the media, there are many fundamental machine learning advances that need to be made to achieve the widespread use of AI in healthcare. Some of the current shortcomings include but are not limited to 1) high error rates in predicting rare critical events (such as cardiac arrest); ii) lack of model robustness to the underlying changes in the data arising due to changes in the policy and practice; iii) lack of the model explainability that makes it hard for the users, e.g. clinicians, to act upon predictions that diverge from their intuition. In my lab, we have started addressing some of these fundamental issues. In this grant I am proposing a research program to make substantial improvements in addressing these questions over the next 5 years.
Classification vs outlier detection The frequency of many critical events, such as cardiac arrest, is usually below 5%. These problems are often incorrectly treated as classification. One of the goals of our work will be to expand the class of generative models, creating models fit for the purpose of outlier detection in healthcare.
Dealing with shifts in the data. Patient populations, care practices, and database systems evolve over time, and yet few works to date have explicitly addressed data shift and evolution as part of the medical record modeling. We showed that there is a simple yet effective mitigation strategy: aggregation of raw features into expert defined clinical concepts. We will develop new robust feature representation techniques and adaptive learning over time to make increase readiness of machine learning (ML) for healthcare deployment in the future.
Explainability. Once developed, translating ML models effectively to practice requires establishing users' trust. We surveyed clinicians to understand their perception of interpretable models. The next step is to integrate these perceptions of explainability into the ML model development. In the program proposed in this grant I will explore counterfactual models to achieve this goal.
Training. My work draws on and affects many disciplines, from computer science to medicine to ethics. Over the last eight years, I have been training HQP coming from a variety of backgrounds, from computer science to biology, from engineering to economics as well as medical professionals in using and developing new AI models for healthcare. It is my goal to supervise and help grow this new and burgeoning field to result in a very highly skilled workforce.
Impact. The amount of available health data has made it clear that machine learning has a great role to play in the future of medicine. The work proposed in this grant will help to build the foundation for responsible AI+healthcare to ensure the realization of the AI potential in this important field.