Grant: Discovery Grants Program - Individual
York University
$32,000.00
Deep Multimodal Learning for Healthcare: Towards Building Multimodal Pathway Health Networks
FY 2023-24
Summary
Human health encompasses many domains (clinical, social, and environmental) influencing well-being in complex ways. Each domain generates data through different modalities and cause-effect relationships. Clinicians process data from multiple modalities when diagnosing and deciding on treatment plans. In contrast, current ML (Machine learning)-enabled assessments and predictions are based on snapshots of the patient's health and have not seen health as a continuous state. Furthermore, most of the current applications of ML in healthcare have addressed narrowly defined tasks using one data modality, such as radiology images or clinical measures extracted from Electronic Health Records (EHR). In theory, ML-enabled health predictive models should be able to use all data sources typically available to clinicians and even those unavailable to most of them (outside of the healthcare settings). The development of multimodal ML models that incorporate data across different modalities, including EHR, wearables, Ambient sensors, imaging, social media text, social determinants of health, and environmental data, is poised to bridge this gap and enable a holistic view of patients' journeys inside and outside the healthcare setting. The main objective of my research is to build a Multimodal Pathway to improve human health. My research program aims to develop theories and algorithms enabling synthesizing and analyzing the complex interplay between multiple aspects of functions related to human health. My research focuses on state-of-the-art techniques rooted in deep learning and transformer models to blend multimodal representation learning and causal inference. Over the next five years, I plan to strengthen my program of research in two critical areas: (1) Building a Unitary Representation from Multimodal Health Data My research explores and builds tools and frameworks to combine information from various modalities and create a Unitary representation. Various data sources such as EHR, images, wearables, ambient sensors, and social media will be explored and integrated into my research. (2) Deep Causal Discovery My research focuses on exploring and discovering objective, unbiased, and actionable insights from multimodal data through transforming representations into causal models, models that demonstrate cause and effect. Through causal discovery, my research build tools to measure the impact of social-determinant of health on measurable health outcomes across different health policies for marginalized populations (including Black, Indigenous, and People Of Color). The impact of my research encompasses contributions to science and applied benefits to human health. The algorithmic outcomes of this research program will result in a comprehensive catalog of novel techniques in multimodal ML. In the long term, my research findings will influence the development of policies and targeted programs that promote health and reduce health disparities.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2023-05063
Name nom
Dolatabadi, Elham
Department département
Institute of Health Policy, Management and Evaluation
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2023
Award amount
32000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2803
Research subject group en
Artificial intelligence
Research subject group fr
Knowledge representation
Research subject en
Intelligence artificielle
Research subject fr
Répresentation des connaissances
Application title
Deep Multimodal Learning for Healthcare: Towards Building Multimodal Pathway Health Networks
Keyword
Ambient and Wearable Sensors; Causal Inference; Deep Learning; Electronic Health Records; Foundation Models; Health Data; Infusion Network; Multimodal learning; Representation Learning; Social determinants of health and health inequalities
Application summary
Human health encompasses many domains (clinical, social, and environmental) influencing well-being in complex ways. Each domain generates data through different modalities and cause-effect relationships. Clinicians process data from multiple modalities when diagnosing and deciding on treatment plans. In contrast, current ML (Machine learning)-enabled assessments and predictions are based on snapshots of the patient's health and have not seen health as a continuous state. Furthermore, most of the current applications of ML in healthcare have addressed narrowly defined tasks using one data modality, such as radiology images or clinical measures extracted from Electronic Health Records (EHR). In theory, ML-enabled health predictive models should be able to use all data sources typically available to clinicians and even those unavailable to most of them (outside of the healthcare settings). The development of multimodal ML models that incorporate data across different modalities, including EHR, wearables, Ambient sensors, imaging, social media text, social determinants of health, and environmental data, is poised to bridge this gap and enable a holistic view of patients' journeys inside and outside the healthcare setting. The main objective of my research is to build a Multimodal Pathway to improve human health. My research program aims to develop theories and algorithms enabling synthesizing and analyzing the complex interplay between multiple aspects of functions related to human health. My research focuses on state-of-the-art techniques rooted in deep learning and transformer models to blend multimodal representation learning and causal inference. Over the next five years, I plan to strengthen my program of research in two critical areas: (1) Building a Unitary Representation from Multimodal Health Data My research explores and builds tools and frameworks to combine information from various modalities and create a Unitary representation. Various data sources such as EHR, images, wearables, ambient sensors, and social media will be explored and integrated into my research. (2) Deep Causal Discovery My research focuses on exploring and discovering objective, unbiased, and actionable insights from multimodal data through transforming representations into causal models, models that demonstrate cause and effect. Through causal discovery, my research build tools to measure the impact of social-determinant of health on measurable health outcomes across different health policies for marginalized populations (including Black, Indigenous, and People Of Color). The impact of my research encompasses contributions to science and applied benefits to human health. The algorithmic outcomes of this research program will result in a comprehensive catalog of novel techniques in multimodal ML. In the long term, my research findings will influence the development of policies and targeted programs that promote health and reduce health disparities.