Grant: Discovery Grants Program - Individual
University of British Columbia
$46,000.00
Analyzing Tests for Correctness, Adequacy, and Effectiveness
FY 2019-20
Summary
Software developers continuously apply code changes to add new features, improve the product, or fix known bugs. For instance, 50% of the Google code base is reported to change every month. In order to ensure that the code base remains healthy, test cases are written to verify that the production code functions as expected. Such tests also serve as continuous regression tests to ensure that previously working functionality still works, when the software evolves. ***Inadequate testing costs the US economy $60 billion annually. A more recent study by Cambridge University finds that software bugs cost the global economy $312 billion per year. The study also found that, on average, software developers spend 50% of their programming time finding and fixing bugs. ***Test cases are meant to form the first line of defence against the introduction of faults in a software product, especially when retesting modified code. Therefore, assessing the quality of a test suite is crucial to ensure that software systems are correctly implemented, which can help to decrease the overall development and maintenance cost.***Test quality can be considered in different perspectives. We propose to study the correctness, adequacy, and effectiveness of test cases, which directly impact the overall test suite quality. The objectives of this research proposal are to (1) study test quality issues related to test correctness, test adequacy, and test effectiveness in practice, (2) explore novel techniques for analyzing and assessing the quality of test cases, (3) and propose new solutions for increasing the quality of test cases directly, and the production code being tested as a result. ***Although the quality of test code is as important as production code, unlike production code, the quality of tests written by developers has not received much attention from the research community thus far, especially when it comes to the correctness of test cases. This proposal seeks to study and analyze the quality of tests in depth. The results of this work will have positive implications for both software testers and developers. The techniques proposed will make test cases more correct, adequate, and effective and as a result the quality of the production code can be improved and the software can be released with much more confident to end-users.**
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2016-04615
Name nom
Mesbah, Ali
Department département
Electrical and Computer Engineering
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2016
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2706
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Software engineering
Research subject fr
Génie logiciel
Installment
inst:4 - 5
Partie
4
Nb partie
5
Application title
Analyzing Tests for Correctness, Adequacy, and Effectiveness
Keyword
NSERC
Application summary
Software developers continuously apply code changes to add new features, improve the product, or fix known bugs. For instance, 50% of the Google code base is reported to change every month. In order to ensure that the code base remains healthy, test cases are written to verify that the production code functions as expected. Such tests also serve as continuous regression tests to ensure that previously working functionality still works, when the software evolves. ***Inadequate testing costs the US economy $60 billion annually. A more recent study by Cambridge University finds that software bugs cost the global economy $312 billion per year. The study also found that, on average, software developers spend 50% of their programming time finding and fixing bugs. ***Test cases are meant to form the first line of defence against the introduction of faults in a software product, especially when retesting modified code. Therefore, assessing the quality of a test suite is crucial to ensure that software systems are correctly implemented, which can help to decrease the overall development and maintenance cost.***Test quality can be considered in different perspectives. We propose to study the correctness, adequacy, and effectiveness of test cases, which directly impact the overall test suite quality. The objectives of this research proposal are to (1) study test quality issues related to test correctness, test adequacy, and test effectiveness in practice, (2) explore novel techniques for analyzing and assessing the quality of test cases, (3) and propose new solutions for increasing the quality of test cases directly, and the production code being tested as a result. ***Although the quality of test code is as important as production code, unlike production code, the quality of tests written by developers has not received much attention from the research community thus far, especially when it comes to the correctness of test cases. This proposal seeks to study and analyze the quality of tests in depth. The results of this work will have positive implications for both software testers and developers. The techniques proposed will make test cases more correct, adequate, and effective and as a result the quality of the production code can be improved and the software can be released with much more confident to end-users.**