Grant: Discovery Launch Supplement

With the fast development and recent success of deep learning (DL) technologies, our society continues to increasingly rely on DL systems including self-driving systems such as Tesla's Autopilot, or general large language models (LLMs) such as ChatGPT. However, such DL systems can have hidden bugs that can lead to serious consequences (even loss of lives in some extreme cases). Hence, my long-term goal is to improve the reliability and robustness of DL systems (e.g., by detecting and repairing bugs in DL systems). My prior research has shown promise in applying differential testing and fuzzing techniques to detect and repair bugs in DL systems as well as using DL models to repair general software bugs. However, there are many unresolved issues such as bug-triggering inputs not being insightful or general automatic repair techniques do not consider the DL models when applied to DL systems. The goal of this proposed research program is to develop new techniques leveraging differential testing and DL model interactions with each other and with DL libraries to improve the reliability of DL systems. This research is composed of three research objectives (ROs): -RO1: We leverage differential testing to validate the performance of multipurpose LLMs by systematically cataloguing such traditional tasks, researching the prior solutions, and comparing LLMs with such solutions. -RO2: We generate insightful bug-triggering inputs-i.e., minimal internal inputs that retain bug-triggering characteristics or propagated external inputs that can trigger the internal error- which will boost the effectiveness of DL API testing tools. -RO3: We develop new approaches to apply fixes to DL libraries by modifying the DL model instead which will save resources by deploying the patched model (easier and faster) while the permanent software fix is being developed (takes longer). This proposed research will help improve the reliability of DL systems and provide systematic testing approaches to remove undesired behavior from DL systems. The proposed research will advance state-of-the-art research by proposing novel techniques to detect bugs and repair DL systems while also provide crucial insights into DL system bugs which can open new research directions on improving and maintaining the reliability of DL systems. Tools from our research would help non-expert DL practitioners (i.e., domain experts) quickly apply LLMs to perform their tasks and DL developers to effectively and efficiently test and repair DL systems. Since Canada has quickly become one of the world's innovation hubs, my research and tools will help local companies reach the world stage. My research plan will train five highly qualified personnel (HQP), who will participate in all aspects of this cutting-edge research.