Grant: Discovery Grants Program - Individual

Recent research in AI has focused on developing versatile models, called foundation models, that can be employed to solve a broader range of tasks than traditional deep learning models. These foundation models exploit self-supervised learning strategies, where large-scale and diverse datasets are used for training without the need for costly annotations. Although such models have caused a paradigm shift in natural language understanding, their impact in medical image analysis has so far been limited. However, such models have unique capabilities that could greatly help advance this field: they can incorporate a small set of examples called prompts to learn completely new tasks without having to retrain, jointly process heterogeneous types of data like images and text by converting them to tokens, and also allow users to interact using plain language. This research program aims to explore and exploit the potential of foundation models based on self-supervised learning to improve medical image analysis. Three complementary lines of research will be investigated, related to adaptability, multi-modal learning, and explainability. First, we will design medical imaging models that can quickly adapt to new tasks and data distributions by performing in-context learning. Second, we will develop novel models which can combine different imaging modalities as well as information from text reports to provide a more accurate analysis. Finally, we will propose models that can better explain their predictions using both plain language and image-based techniques. The AI-based solutions proposed by this research have the potential to automate some of the tasks performed by clinicians, freeing up their time to give patients a better care. By enhancing the adaptability, the ability to process multi-modal data, and the explainability of AI solutions for medical imaging, this research also can contribute to improve the applicability, user adoption, and safety of such tools in practice.