Grant: Discovery Grants Program - Individual

The innovative applications of machine learning (ML) must be robust and fair in this new era of artificial intelligence. Therefore, it is primordial to understand current ML models' vulnerabilities to build models that cannot be easily manipulated or corrupted by malicious attackers and operate reliably and fairly. Unfortunately, there is a big gap between what is delivered by current ML approaches that attempt to reproduce the human decision-making process. Such approaches can achieve low error rates on many supervised learning tasks where large amounts of data are available, but they neglect that humans are multisensorial and rely on information from different sensory modalities to make decisions. Multimodal ML models that use multiple input sources to benefit from complementary and shared information are being increasingly devised for real-life applications, from self-driving vehicles that combine the information of multiple sensors to applications in health that analyze multiple cues from persons. Therefore, building robust multimodal models are crucial for advancing the knowledge in ML and for achieving reliable deployment in several safety-critical applications. Although multimodal ML models are reaching state-of-the-art performance in several tasks, relatively little is known about the robustness of such models. Little effort has been put into robust multimodal ML models that deal with several data modalities, such as image, audio, text, and physiological signals. Notably, on tasks where personal data is available, besides the robustness, assuring that decisions made by ML models are not skewed toward a particular group of people is also a main concern. Furthermore, few studies have evaluated the fairness of multimodal ML models and proposed measures to mitigate bias in such models. Therefore, it is imperative to investigate whether they are robust and fair in worst-case scenarios and to develop strategies to build accurate, robust, and fair multimodal ML models. To achieve such a goal, this research program has three main objectives: 1- understanding and characterization of failure modes of multimodal ML models and design of robust approaches that provide a good trade-off between accuracy and robustness; 2- investigating the potential of adversarial attacks to produce perturbed inputs that cause multimodal ML models to make unfair decisions, and design approaches for bias mitigation; 3 - investigating the interaction between accuracy, robustness, and fairness and develop novel approaches that provide a good trade-off among them, in domains where all these aspects are critical. Therefore, this research program is fundamental to paving the way to developing robust and fair multimodal ML applications that can deal with multisensorial real-life data for high-stakes applications. That will help Canadian companies to have more confident corporate decision-making and to maintain their competitive advantage.