Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Calgary

$24,000.00

Improving the consistency and speed of qualitative data analysis to support software engineering researchers and requirements engineering practitioners

FY 2023-24

Summary
The systematic analysis of textual data through qualitative data analysis (QDA) is used by researchers addressing questions involving real-world phenomena in context. The technique has also been proposed for industry applications, where consistency, completeness and traceability of the analysis are required. For example, in requirements engineering (RE), QDA has been proposed as a way of systematically generating specifications which are complete and which don't rely on hidden, implicit knowledge possessed only by the analyst. However, because QDA is designed to thoroughly explore the problem under consideration, it results in an unwieldy number of concepts. This means that analysts must adopt a number of techniques to ensure consistency, which are time-consuming and may still fail to ensure consistency within a single analysis, or between group members working on the same analysis. QDA is time-consuming, which affects industry use. Consistency and time-cost both severely limit adoption of QDA. Software engineering researchers have explored extending computer-assisted qualitative data analysis software (CAQDAS) by using machine learning to propose annotations or categories for later analysis, based on initial analysis. However, none of these approaches has gone beyond the domain of software engineering, or achieved results which would lead to adoption of the technique. I propose to tackle consistency and time cost through the assistive use of natural language processing. The proposal focuses on two audiences: qualitative researchers and practitioners working in requirements engineering. RE is one of the fields where QDA has been proposed as a means of ensuring completeness, consistency, and pre-requirements specification traceability. The proposal includes exploratory research into other limitations and barriers for both academic and practitioner audiences. Ultimately, the usefulness of the proposed solution is evaluated in terms of the quality of the analyses, and the experiences of the users. I expect the results to exceed the limits of the exploratory studies due to an iterative process with extensive stakeholder engagement. This is part of a broader program to identify and address limitations in CAQDAS systems, and barriers to the use of QDA by industry. Beyond the scope of this proposal, I plan to seek solutions to other problems preventing QDA use, and examine the solutions in other areas of software engineering which would benefit from the rigor of QDA. Industry funding will be sought once initial results have demonstrated the usefulness of the approach. The proposed research will impact qualitative researchers and industry in Canada and beyond, by reducing the problems associated with using QDA. The research will be supported by two PhD students, two MScs student, and one BSc students. Due to the practical application of the research, HQP will be able to acquire both research skills and industry experience.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2021-02405
Name nom
Barcomb, Ann
Department département
Schulich School of Engineering
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Improving the consistency and speed of qualitative data analysis to support software engineering researchers and requirements engineering practitioners
Keyword
computer-assisted qualitative data analysis; empirical software engineering; human-centered computing; natural language processing; qualitative data analysis; requirements engineering; research methods; text analytics
Application summary
The systematic analysis of textual data through qualitative data analysis (QDA) is used by researchers addressing questions involving real-world phenomena in context. The technique has also been proposed for industry applications, where consistency, completeness and traceability of the analysis are required. For example, in requirements engineering (RE), QDA has been proposed as a way of systematically generating specifications which are complete and which don't rely on hidden, implicit knowledge possessed only by the analyst. However, because QDA is designed to thoroughly explore the problem under consideration, it results in an unwieldy number of concepts. This means that analysts must adopt a number of techniques to ensure consistency, which are time-consuming and may still fail to ensure consistency within a single analysis, or between group members working on the same analysis. QDA is time-consuming, which affects industry use. Consistency and time-cost both severely limit adoption of QDA. Software engineering researchers have explored extending computer-assisted qualitative data analysis software (CAQDAS) by using machine learning to propose annotations or categories for later analysis, based on initial analysis. However, none of these approaches has gone beyond the domain of software engineering, or achieved results which would lead to adoption of the technique. I propose to tackle consistency and time cost through the assistive use of natural language processing. The proposal focuses on two audiences: qualitative researchers and practitioners working in requirements engineering. RE is one of the fields where QDA has been proposed as a means of ensuring completeness, consistency, and pre-requirements specification traceability. The proposal includes exploratory research into other limitations and barriers for both academic and practitioner audiences. Ultimately, the usefulness of the proposed solution is evaluated in terms of the quality of the analyses, and the experiences of the users. I expect the results to exceed the limits of the exploratory studies due to an iterative process with extensive stakeholder engagement. This is part of a broader program to identify and address limitations in CAQDAS systems, and barriers to the use of QDA by industry. Beyond the scope of this proposal, I plan to seek solutions to other problems preventing QDA use, and examine the solutions in other areas of software engineering which would benefit from the rigor of QDA. Industry funding will be sought once initial results have demonstrated the usefulness of the approach. The proposed research will impact qualitative researchers and industry in Canada and beyond, by reducing the problems associated with using QDA. The research will be supported by two PhD students, two MScs student, and one BSc students. Due to the practical application of the research, HQP will be able to acquire both research skills and industry experience.