Grant: Discovery Grants Program - Individual
Dalhousie University
$29,000.00
Mining, Intelligence and Automation in Tackling Machine-Learning Bugs
EF 2023-24
Résumé
Motivation and Objectives: According to Fortune Business Insights, Machine Learning (ML) software is projected to have a global market share of $117 billion by 2027. It has found applications in major areas including healthcare, transportation, and business analytics. However, the technology is yet to mature and can be unreliable. Bugs in ML software are highly complex, hard to solve due to their data-driven, non-deterministic nature and have potentially deadly consequences (e.g., the fatal crash of Uber's self-driving car). The standard procedure for correcting bugs is labour-intensive and inefficient, which takes up ~50% of a developer's time. The majority of this time is spent finding and understanding the faulty code before making actual code changes. To date, many approaches were designed to find and solve bugs in traditional software, but most are not accurate enough and are inadequate for application to ML software. My research program aims to design intelligent frameworks for understanding, finding, and reproducing bugs in ML software. Research Plan and Methodology: This proposal encompasses three complementary activities: (1) advancing the current understanding of ML bugs, (2) finding bugs in ML software, and (3) reproducing the identified bugs for a reliable diagnosis. First, we will construct a large dataset from ML applications found at GitHub to systematically study the characteristics and central challenges of ML bugs and analyze the effectiveness of traditional debugging solutions to these bugs. Second, we will design an intelligent framework that can (a) detect faulty components using intelligent Information Retrieval methods, (b) detect the faulty code within these components using their static properties and dynamic behaviours, and (c) complement these results with meaningful explanations (e.g., type of bug). Third, we will design an intelligent framework that can (a) help a developer understand how a bug might trigger, and (b) deliver appropriate test cases to reproduce the identified bugs in ML software using reinforcement learning and a technology sandbox. Novelty and Expected Significance: This research program has three novel aspects: (a) intelligent debugging supports for ML software, (b) extension of developer's cognitive abilities with machine intelligence, and (c) enrichment of tools' results with complementary information. It will advance the current state of research for cost-effective debugging and will also benefit parallel practices such as change management. My research will also produce tools that will be adopted by industry, such as through my collaborations with Mozilla Corporation and Canadian software companies. By supporting developers in solving ML bugs efficiently and by providing high-quality training to students in an area of acute need, this program will thus assist in the development of safe, reliable machine-learning software and significantly contribute to the Canadian economy.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Nova Scotia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2021-03236
Name nom
Rahman, Mohammad Masudur
Department département
Faculty of Computer Science
Organization id
79
Institution établissement
Dalhousie University
Province en
Nova Scotia
Province fr
Nouvelle-Écosse
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2705
Research subject group en
Information technology
Research subject group fr
Software and development
Research subject en
Technologie de l'information
Research subject fr
Logiciels et développement
Application title
Mining, Intelligence and Automation in Tackling Machine-Learning Bugs
Keyword
bug localization; bug reproduction; information retrieval; intelligent automation; machine-learning bugs; mining software repositories; multimodal study; query reformulation; software bugs and errors; software debugging
Application summary
Motivation and Objectives: According to Fortune Business Insights, Machine Learning (ML) software is projected to have a global market share of $117 billion by 2027. It has found applications in major areas including healthcare, transportation, and business analytics. However, the technology is yet to mature and can be unreliable. Bugs in ML software are highly complex, hard to solve due to their data-driven, non-deterministic nature and have potentially deadly consequences (e.g., the fatal crash of Uber's self-driving car). The standard procedure for correcting bugs is labour-intensive and inefficient, which takes up ~50% of a developer's time. The majority of this time is spent finding and understanding the faulty code before making actual code changes. To date, many approaches were designed to find and solve bugs in traditional software, but most are not accurate enough and are inadequate for application to ML software. My research program aims to design intelligent frameworks for understanding, finding, and reproducing bugs in ML software. Research Plan and Methodology: This proposal encompasses three complementary activities: (1) advancing the current understanding of ML bugs, (2) finding bugs in ML software, and (3) reproducing the identified bugs for a reliable diagnosis. First, we will construct a large dataset from ML applications found at GitHub to systematically study the characteristics and central challenges of ML bugs and analyze the effectiveness of traditional debugging solutions to these bugs. Second, we will design an intelligent framework that can (a) detect faulty components using intelligent Information Retrieval methods, (b) detect the faulty code within these components using their static properties and dynamic behaviours, and (c) complement these results with meaningful explanations (e.g., type of bug). Third, we will design an intelligent framework that can (a) help a developer understand how a bug might trigger, and (b) deliver appropriate test cases to reproduce the identified bugs in ML software using reinforcement learning and a technology sandbox. Novelty and Expected Significance: This research program has three novel aspects: (a) intelligent debugging supports for ML software, (b) extension of developer's cognitive abilities with machine intelligence, and (c) enrichment of tools' results with complementary information. It will advance the current state of research for cost-effective debugging and will also benefit parallel practices such as change management. My research will also produce tools that will be adopted by industry, such as through my collaborations with Mozilla Corporation and Canadian software companies. By supporting developers in solving ML bugs efficiently and by providing high-quality training to students in an area of acute need, this program will thus assist in the development of safe, reliable machine-learning software and significantly contribute to the Canadian economy.