Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

École Polytechnique de Montréal

$41,000.00

Improving the Quality Assurance of Machine-Learning Software Applications

FY 2021-22

Summary
Machine learning (ML) is increasingly deployed in large-scale and critical systems thanks to recent breakthroughs in deep learning and reinforcement learning. We are now using software applications powered by ML in critical aspects of our daily lives; from finance, energy, to health and transportation. The economic benefits of Machine-Learning Software Applications (MLSA) and Artificial Intelligence (AI) in general is forecast to surpass USD 8.81 Billion by 2022. However, ensuring the quality assurance of MLSA is still very challenging as evidenced by the recent deadly incident caused by the $47-million Michigan Integrated Data Automated System (MiDAS), or the Uber's self-driving car that ran into a pedestrian even though the car's sensors detected her presence. The MLSA running the Uber's car reportedly considered the detection of the pedestrian as a "false positive". The main reason behind the difficulty to ensure quality in MLSA is the shift in the development paradigm induced by ML and AI. Traditionally, software systems are constructed deductively, by writing down the rules that govern the behavior of the system as program code. However, with ML, these rules are inferred from training data (i.e., the requirements are generated inductively). This paradigm shift in application development makes it difficult to reason about the behavior of software systems with ML components, resulting in systems that are intrinsically challenging to test and verify. A defect in a MLSA may come from its training data, program code, execution environment, or third-party frameworks (e.g., TensorFlow). Also, ML models must be retrained and evolved constantly to cope with changes in users' behaviors, model drift, or adversarial interactions for example, hence the necessity to architect them in a way that minimizes the cost of these frequent models changes on their overall maintenance and evolution. Current existing software development techniques must be revisited and adapted to this new reality. The goal of this research program is to develop techniques and tools to support quality assurance activities for MLSA systems, given that they do not have (complete) specifications or even source code corresponding to some of their critical behaviors (some MLSA rely on proprietary third-party libraries like Intel Math Kernel Library for many critical operations). Through this research program, my students and I will identify good and bad development practices that can impede the maintenance and the reliability of MLSA. I will also develop techniques and tools to help developers detect and correct errors in MLSA, both at design and implementation levels.
Program
Discovery Grants Program - Individual
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2019-06956
Name nom
Khomh, Foutse
Department département
Génie informatique
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2019
Award amount
41000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2706
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Software engineering
Research subject fr
Génie logiciel
Installment
inst:3
Partie
1
Nb partie
5
Application title
Improving the Quality Assurance of Machine-Learning Software Applications
Keyword
NSERC
Application summary
Machine learning (ML) is increasingly deployed in large-scale and critical systems thanks to recent breakthroughs in deep learning and reinforcement learning. We are now using software applications powered by ML in critical aspects of our daily lives; from finance, energy, to health and transportation. The economic benefits of Machine-Learning Software Applications (MLSA) and Artificial Intelligence (AI) in general is forecast to surpass USD 8.81 Billion by 2022. However, ensuring the quality assurance of MLSA is still very challenging as evidenced by the recent deadly incident caused by the $47-million Michigan Integrated Data Automated System (MiDAS), or the Uber's self-driving car that ran into a pedestrian even though the car's sensors detected her presence. The MLSA running the Uber's car reportedly considered the detection of the pedestrian as a "false positive". The main reason behind the difficulty to ensure quality in MLSA is the shift in the development paradigm induced by ML and AI. Traditionally, software systems are constructed deductively, by writing down the rules that govern the behavior of the system as program code. However, with ML, these rules are inferred from training data (i.e., the requirements are generated inductively). This paradigm shift in application development makes it difficult to reason about the behavior of software systems with ML components, resulting in systems that are intrinsically challenging to test and verify. A defect in a MLSA may come from its training data, program code, execution environment, or third-party frameworks (e.g., TensorFlow). Also, ML models must be retrained and evolved constantly to cope with changes in users' behaviors, model drift, or adversarial interactions for example, hence the necessity to architect them in a way that minimizes the cost of these frequent models changes on their overall maintenance and evolution. Current existing software development techniques must be revisited and adapted to this new reality. The goal of this research program is to develop techniques and tools to support quality assurance activities for MLSA systems, given that they do not have (complete) specifications or even source code corresponding to some of their critical behaviors (some MLSA rely on proprietary third-party libraries like Intel Math Kernel Library for many critical operations). Through this research program, my students and I will identify good and bad development practices that can impede the maintenance and the reliability of MLSA. I will also develop techniques and tools to help developers detect and correct errors in MLSA, both at design and implementation levels.