Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Calgary

$32,000.00

Time Matters: Towards Unified Explainable Machine Learning Models for High-Dimensional and Multimodal Medical Image Data with Temporal Components

FY 2023-24

Summary
Medical images are widely used in diagnostics and research and they are often automatically analyzed by machine learning (ML) tools. However, their ML-based analysis poses many challenges: (1) High spatial dimensionality/resolution (e.g., a 3D brain magnetic resonance imaging - MRI - scan contains millions of image elements), (2) multimodality (e.g., multiple MRI sequences available for a subject), and (3) temporal components (e.g., scans repeated over time to assess changes). This results in a massive amount of heterogeneous data that should be processed in a combined way, which requires specialized and efficient methods to make full use of the information. Such ML models not only need to be accurate, but explainability of their decisions is also important to enhance trust by clinicians and patients and to aid reliable image-driven biomarker discovery. Existing methods for medical image analysis are mostly task- and modality-specific, unable to jointly handle multiple modalities with temporal components, and lack built-in explainability capabilities. This results in underutilized diagnostic and research potentials, given the ever increasing amounts of imaging and also non-imaging data (e.g., biosignals from wearables) being acquired. The long-term goal of this research program is to develop novel ML methods for a holistic analysis of all data available from precision medicine measurements. Over the next 5 years, we will develop innovative, self-explainable generative discriminative ML models that that do not suffer from the mentioned restrictions and tackle major challenges related to the analysis of high-dimensional, multimodal, temporal medical image data in a unified way, which has never been attempted before at this scale. Our novel models will have the potential to improve image-driven diagnostics and research and our work will further strengthen Canada's research capacities in the area of biomedical ML. While standard discriminative ML models map images to a diagnostic score, we will develop models that combine discriminative capabilities (e.g., disease classification) with generative abilities as they will also be able to synthesize images given a score and to reverse their decision-making process. This will equip them with built-in longitudinal forecasting and self-explainability capacities as an important step towards trustworthy ML and flexible multi-purpose models in medical image analysis. Their development will be achieved via three specific aims. First, we will develop novel data compression as well as multimodal and temporal processing methods to build powerful, efficient generative classifiers/regressors. Their generative capabilities will then be used to design novel visual, interactive self-explainability techniques that provide insights into the decision-making process. Finally, all methods will be integrated into a publicly available software package to enable their use in other projects and also other application areas.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2023-03532
Name nom
Wilms, Matthias
Department département
Department of Pediatrics
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2023
Award amount
32000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1102
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Biomedical engineering
Area of application fr
Génie biomédical
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Time Matters: Towards Unified Explainable Machine Learning Models for High-Dimensional and Multimodal Medical Image Data with Temporal Components
Keyword
biomedical image analysis; explainable AI; machine learning; multimodal data; neuroimaging; spatio-temporal data
Application summary
Medical images are widely used in diagnostics and research and they are often automatically analyzed by machine learning (ML) tools. However, their ML-based analysis poses many challenges: (1) High spatial dimensionality/resolution (e.g., a 3D brain magnetic resonance imaging - MRI - scan contains millions of image elements), (2) multimodality (e.g., multiple MRI sequences available for a subject), and (3) temporal components (e.g., scans repeated over time to assess changes). This results in a massive amount of heterogeneous data that should be processed in a combined way, which requires specialized and efficient methods to make full use of the information. Such ML models not only need to be accurate, but explainability of their decisions is also important to enhance trust by clinicians and patients and to aid reliable image-driven biomarker discovery. Existing methods for medical image analysis are mostly task- and modality-specific, unable to jointly handle multiple modalities with temporal components, and lack built-in explainability capabilities. This results in underutilized diagnostic and research potentials, given the ever increasing amounts of imaging and also non-imaging data (e.g., biosignals from wearables) being acquired. The long-term goal of this research program is to develop novel ML methods for a holistic analysis of all data available from precision medicine measurements. Over the next 5 years, we will develop innovative, self-explainable generative discriminative ML models that that do not suffer from the mentioned restrictions and tackle major challenges related to the analysis of high-dimensional, multimodal, temporal medical image data in a unified way, which has never been attempted before at this scale. Our novel models will have the potential to improve image-driven diagnostics and research and our work will further strengthen Canada's research capacities in the area of biomedical ML. While standard discriminative ML models map images to a diagnostic score, we will develop models that combine discriminative capabilities (e.g., disease classification) with generative abilities as they will also be able to synthesize images given a score and to reverse their decision-making process. This will equip them with built-in longitudinal forecasting and self-explainability capacities as an important step towards trustworthy ML and flexible multi-purpose models in medical image analysis. Their development will be achieved via three specific aims. First, we will develop novel data compression as well as multimodal and temporal processing methods to build powerful, efficient generative classifiers/regressors. Their generative capabilities will then be used to design novel visual, interactive self-explainability techniques that provide insights into the decision-making process. Finally, all methods will be integrated into a publicly available software package to enable their use in other projects and also other application areas.