Grant: Discovery Launch Supplement
University of Calgary
$12,500.00
Next Generation of Methods and Models for Self-supervised Multimodal Deep Learning for Biomedical Data Analysis
FY 2024-25
Summary
Deep learning, a type of artificial intelligence (AI), is becoming increasingly popular in the biomedical field. To work effectively, it often requires large amounts of data with specific labels. However, manually labeling this data is challenging, especially in contexts where various medical images such as CT scans, MRIs, and echocardiograms are used to record information, creating multi-type (or multimodal) datasets. While humans can easily combine and understand information from multiple sources, current AI approaches mostly deal with a single data type, limiting their potential.
My long-term vision is to achieve human-level proficiency in seamlessly integrating multi-sensory data, thereby equipping AI systems to learn and interpret intricate concepts effectively. This requires advancing multimodal deep learning and learning with minimal supervision. Aligned with this vision, my short-term objectives are as follows:
A temporal image is a sequence of images as time passes. Simulating temporal images is challenging, especially in domains without large datasets, as AI methods often struggle to maintain temporal consistency. My research will pioneer physics-aware simulations of medical images by utilizing modern game engine technologies, such as Unreal Engine, and generative models. Physics-aware simulations using game engines will offer temporal consistency and novel temporal generative models will enhance image realism.
While fruit flies possess a relatively simple neural network compared to more complex organisms, they are known for their adept flying and sophisticated foraging strategies. Fruit flies perform complex tasks with limited neural circuitry. The publication of a detailed map of the neural connections of a fruit fly in 2023 allows us to use computational techniques to uncover patterns within these connections. We will design efficient and high-performing neural networks by leveraging insights from these patterns.
Utilizing complex multimodal temporal images for analyses demands extensive annotations of each data type across all time steps and thus poses challenges for traditional supervised training methods. My research aims to alleviate these challenges by introducing learning paradigms designed for multimodal temporal data, capitalizing on the inherent structures and patterns present within individual data types and the rich correlations and connections among data modalities.
These objectives-enhancing data accuracy, consistency, and comprehensiveness through physics-aware simulations; emulating the efficient sensory integration observed in humans; and tapping into the intrinsic human capacity for contextual learning-will advance AI toward human-level multi-sensory data integration. The innovation derived from this research will promote the widespread adoption of deep learning methods across key sectors of the Canadian economy.
Program
Discovery Launch Supplement
Recipient
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
DGECR-2024-00319
Name nom
Maleki, Farhad
Department département
Computer Science
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
12500
Program id
DGECR
Program name en
Discovery Launch Supplement
Program name fr
Supplément Tremplin vers la découverte
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2708
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Image processing
Research subject fr
Traitement des images
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Next Generation of Methods and Models for Self-supervised Multimodal Deep Learning for Biomedical Data Analysis
Keywords
image processing; biomedical data analysis; biomedical image analysis; video processing; multimodal data analsysis; synthetic data
Application summary
Deep learning, a type of artificial intelligence (AI), is becoming increasingly popular in the biomedical field. To work effectively, it often requires large amounts of data with specific labels. However, manually labeling this data is challenging, especially in contexts where various medical images such as CT scans, MRIs, and echocardiograms are used to record information, creating multi-type (or multimodal) datasets. While humans can easily combine and understand information from multiple sources, current AI approaches mostly deal with a single data type, limiting their potential.
My long-term vision is to achieve human-level proficiency in seamlessly integrating multi-sensory data, thereby equipping AI systems to learn and interpret intricate concepts effectively. This requires advancing multimodal deep learning and learning with minimal supervision. Aligned with this vision, my short-term objectives are as follows:
A temporal image is a sequence of images as time passes. Simulating temporal images is challenging, especially in domains without large datasets, as AI methods often struggle to maintain temporal consistency. My research will pioneer physics-aware simulations of medical images by utilizing modern game engine technologies, such as Unreal Engine, and generative models. Physics-aware simulations using game engines will offer temporal consistency and novel temporal generative models will enhance image realism.
While fruit flies possess a relatively simple neural network compared to more complex organisms, they are known for their adept flying and sophisticated foraging strategies. Fruit flies perform complex tasks with limited neural circuitry. The publication of a detailed map of the neural connections of a fruit fly in 2023 allows us to use computational techniques to uncover patterns within these connections. We will design efficient and high-performing neural networks by leveraging insights from these patterns.
Utilizing complex multimodal temporal images for analyses demands extensive annotations of each data type across all time steps and thus poses challenges for traditional supervised training methods. My research aims to alleviate these challenges by introducing learning paradigms designed for multimodal temporal data, capitalizing on the inherent structures and patterns present within individual data types and the rich correlations and connections among data modalities.
These objectives-enhancing data accuracy, consistency, and comprehensiveness through physics-aware simulations; emulating the efficient sensory integration observed in humans; and tapping into the intrinsic human capacity for contextual learning-will advance AI toward human-level multi-sensory data integration. The innovation derived from this research will promote the widespread adoption of deep learning methods across key sectors of the Canadian economy.