Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Calgary

$24,000.00

Using structural priors to learn transferable representations

FY 2024-25

Summary
General context: In recent years, deep learning has enabled impressive progress on several tasks in computer vision, natural language processing, and other core artificial intelligence domains. This has led to more effort towards more complex and realistic applications, often involving multiple modalities (e.g. images and text). One challenge machine learning practitioners face is that often trained models do not perform well on new data. In particular, the problem of out-of-distribution (OOD) generalization, where the assumption is that new data is not drawn from the same distribution as the training data. For instance, in medical imaging, a model could be trained on data from several hospitals but will be used to analyze images from other hospitals with different devices and patient distributions. This problem is caused by models latching onto spurious correlations or "shortcuts" in the training data, which do not correspond to a meaningful solution. Objectives: Our research program is concerned with the development of flexible architectural priors that improve OOD generalization in real-world tasks (e.g. across patients in medical imaging, from simulation to a real robot in robotics, or across satellite images from different regions in remote sensing). To be able to develop and test novel architectures for OOD generalization, we focus on the following three domains in our short-term goals: (i) learning of disentangled representations in multimodal learning, (ii) learning generalizable skills in reinforcement learning agents with a focus on capturing dynamics, and (iii) few-shot learning as a test-bed for transferability of multimodal features. Progress on all of these domains with a similar methodology will be a significant step towards the long-term objective. In each of these domains, we will investigate two groups of architectural priors: (i) knowledge transfer between modalities (e.g. knowledge distillation or feature modulation) and (ii) hierarchical representations (e.g. structured model architectures involving multiplicative interactions). Significance: Poor OOD generalization is an obstacle in the deployment of deep learning methods on many real-world tasks. Solving the problem requires capturing domain structure in the model. An architecture that can leverage multiple modalities and prior knowledge from domain experts, can be guided towards a more general solution, avoiding many of the shortcuts learned by current architectures. The potential impact of the proposed research is profound since shortcut learning makes models prone to reflecting many forms of bias, that are inherent in datasets or architectural details of commonly-used methods. Reducing bias promises the flexibility of improving personalized solutions, such as in medical diagnostics or education. In domains where data collection is expensive (e.g. medical imaging, robotics, high-resolution remote sensing), structured models can be trained using fewer examples.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2021-04086
Name nom
Ebrahimi Kahou, Samira
Department département
Electrical and Software Engineering - Electrical and Software Engineering
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2021
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Using structural priors to learn transferable representations
Keywords
deep learning; multi-modal learning; generalization; reinforcement learning; meta-learning; computer vision; representation learning
Application summary
General context: In recent years, deep learning has enabled impressive progress on several tasks in computer vision, natural language processing, and other core artificial intelligence domains. This has led to more effort towards more complex and realistic applications, often involving multiple modalities (e.g. images and text). One challenge machine learning practitioners face is that often trained models do not perform well on new data. In particular, the problem of out-of-distribution (OOD) generalization, where the assumption is that new data is not drawn from the same distribution as the training data. For instance, in medical imaging, a model could be trained on data from several hospitals but will be used to analyze images from other hospitals with different devices and patient distributions. This problem is caused by models latching onto spurious correlations or "shortcuts" in the training data, which do not correspond to a meaningful solution. Objectives: Our research program is concerned with the development of flexible architectural priors that improve OOD generalization in real-world tasks (e.g. across patients in medical imaging, from simulation to a real robot in robotics, or across satellite images from different regions in remote sensing). To be able to develop and test novel architectures for OOD generalization, we focus on the following three domains in our short-term goals: (i) learning of disentangled representations in multimodal learning, (ii) learning generalizable skills in reinforcement learning agents with a focus on capturing dynamics, and (iii) few-shot learning as a test-bed for transferability of multimodal features. Progress on all of these domains with a similar methodology will be a significant step towards the long-term objective. In each of these domains, we will investigate two groups of architectural priors: (i) knowledge transfer between modalities (e.g. knowledge distillation or feature modulation) and (ii) hierarchical representations (e.g. structured model architectures involving multiplicative interactions). Significance: Poor OOD generalization is an obstacle in the deployment of deep learning methods on many real-world tasks. Solving the problem requires capturing domain structure in the model. An architecture that can leverage multiple modalities and prior knowledge from domain experts, can be guided towards a more general solution, avoiding many of the shortcuts learned by current architectures. The potential impact of the proposed research is profound since shortcut learning makes models prone to reflecting many forms of bias, that are inherent in datasets or architectural details of commonly-used methods. Reducing bias promises the flexibility of improving personalized solutions, such as in medical diagnostics or education. In domains where data collection is expensive (e.g. medical imaging, robotics, high-resolution remote sensing), structured models can be trained using fewer examples.