Grant: Discovery Grants Program - Individual
Carleton University
$29,000.00
Machine learning methods that make sense to humans
EF 2021-22
Résumé
Machine learning methods are widely used in predictions and decision making, and people are affected more than ever by these algorithmic decisions. Our society has become significantly influenced by the use of AI and this in turn has raised a lot of concerns about AI's transparency, bias and fairness. Research directed towards the interpretability of AI has become a topic of intense interest in recent years. My research program on interpretability will proceed in two fundamental directions. The first is to develop novel methods that are designed with interpretability in mind. Despite the recent popularity of deep learning methods and their successful application in areas such as object recognition and machine translation, many other areas -especially in finance, insurance and, to some extent health- have been hesitant to benefit from advances in deep learning, mainly because of issues rooted in the transparency, trust, bias and safety of AI. Those are areas where most data are structured (as opposed to unstructured data such as audio and video). Decisions made based on a smaller number of features are more likely to be interpretable by human. I will revisit feature selection and develop methods that allow explaining the decision by referring to similar samples in the training set. This mimics how physicians often explain their decision by referring to similar cases in the past. I will further extend the idea to learning deep embedding spaces that can provide such explanations. Alternatively, I propose an approach for generating explanations from a semantic space. The second direction of the program seeks to develop model-agnostic methods for improving the interpretability of an existing black-box model. Here, interpretability may be seen as a post-hoc processing to shed light on how the model came to its decision, and if, what the model has learned makes sense to human. Along this line, I would like to examine methods that involve human-friendly concepts. Humans often explain their decisions by referring to concepts that are intuitive to humans but are not directly related to the prediction problem. Domain experts often have concepts that they care about and would like to examine if a machine learning model pays attention to them. International Data Corporation estimates that worldwide investment in AI will have grown from US$24 billion in 2018 to US$77.7 billion by 2022. The widespread use of AI implies an increasingly significant impact on society. The program results will help establish trust in AI and mitigate some of the ethical concerns about deploying AI.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2020-06720
Name nom
Komeili, Majid
Department département
Computer Science
Organization id
23
Institution établissement
Carleton University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2020
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Installment
inst:2
Partie
1
Nb partie
5
Application title
Machine learning methods that make sense to humans
Keyword
NSERC
Application summary
Machine learning methods are widely used in predictions and decision making, and people are affected more than ever by these algorithmic decisions. Our society has become significantly influenced by the use of AI and this in turn has raised a lot of concerns about AI's transparency, bias and fairness. Research directed towards the interpretability of AI has become a topic of intense interest in recent years. My research program on interpretability will proceed in two fundamental directions. The first is to develop novel methods that are designed with interpretability in mind. Despite the recent popularity of deep learning methods and their successful application in areas such as object recognition and machine translation, many other areas -especially in finance, insurance and, to some extent health- have been hesitant to benefit from advances in deep learning, mainly because of issues rooted in the transparency, trust, bias and safety of AI. Those are areas where most data are structured (as opposed to unstructured data such as audio and video). Decisions made based on a smaller number of features are more likely to be interpretable by human. I will revisit feature selection and develop methods that allow explaining the decision by referring to similar samples in the training set. This mimics how physicians often explain their decision by referring to similar cases in the past. I will further extend the idea to learning deep embedding spaces that can provide such explanations. Alternatively, I propose an approach for generating explanations from a semantic space. The second direction of the program seeks to develop model-agnostic methods for improving the interpretability of an existing black-box model. Here, interpretability may be seen as a post-hoc processing to shed light on how the model came to its decision, and if, what the model has learned makes sense to human. Along this line, I would like to examine methods that involve human-friendly concepts. Humans often explain their decisions by referring to concepts that are intuitive to humans but are not directly related to the prediction problem. Domain experts often have concepts that they care about and would like to examine if a machine learning model pays attention to them. International Data Corporation estimates that worldwide investment in AI will have grown from US$24 billion in 2018 to US$77.7 billion by 2022. The widespread use of AI implies an increasingly significant impact on society. The program results will help establish trust in AI and mitigate some of the ethical concerns about deploying AI.