Grant: Discovery Grants Program - Individual

Despite the great success of deep neural networks (DNNs) in a wide spectrum of applications, DNN models are often opaque: they provide limited insight for the understanding of model decisions and are typically treated as "black boxes". This non-transparency limits their scope of application, as many domains, e.g., finance, military, and health care, require strong interpretability and trustworthiness. To address this issue, a great deal of effort has been made in eXplainable Artificial Intelligence (XAI). For example, the U.S. Defense Advanced Research Projects Agency (DRAPA) initiated a challenge to help warfighters understand and trust AI machine partners in 2017; Big Tech companies like Google, Microsoft, and IBM launched their respective XAI toolkits in 2019; IEEE approved the first XAI developing standard in online banking in 2020. The proposed research focuses on enabling interpretability of DNN models in natural language processing (NLP), which has been less studied compared to XAI in other domains such as enabling interpretability in computer vision (CV). This task is particularly difficult as most DNN-based NLP (Neural NLP) methods consider natural language tokens as inputs, whose discrete nature makes existing solutions in CV, which consume continuous pixels in images as input, not directly applicable. Moreover, the understanding of natural language presents unique challenges such as sarcasm, slang, and cultural effect in the traditional learning framework, further increasing the difficulty in interpretation. The long-term objective of the proposed program is to enable trustworthy machine learning (ML) in the context of NLP, and create frameworks and methods that can help optimize ML models for improved interpretability. The short-term objective is to develop systematic approaches and attain interpretability at different stages of machine learning, i.e., the input level, the processing level, and the evaluation level. Specifically, this research is devoted to answering the following questions: (1) What language features are most important to prediction? (2) What do DNNs learn in NLP? (3) How to evaluate the quality of explanations? Meanwhile, a total of 12 HQPs (2 PhDs, 5 Master's, and 5 undergraduates) will be trained over the course of the research program. The proposed research will advance the state of the art in Neural NLP interpretation, and will enable further exploration of this important area by the academic community. It will also help organizations and individual users build trust in the model, perform model diagnosis and debugging, and enable wider deployment of decision-making applications in different domains such as law enforcement, market forecasting, and medical diagnoses.