Grant: Discovery Grants Program - Individual
École Polytechnique de Montréal
$24,000.00
Towards Systematic and Cost-Effective Monitoring of Large-Scale Software Systems
EF 2021-22
Résumé
Failures of large-scale software systems can have significant impacts on our lives and safety, as well as on the economy and security of our society. For example, failures of the Canada Revenue Agency (CRA) website in March 2017 and March 2019 blocked many Canadians from filing their taxes for days. To detect and address such failures quickly and reduce their impact on users, it is crucial to gain an understanding of the internal state and runtime behavior of these systems. Software observability (i.e., the extent to which the internal state of a system can be inferred) allows software engineers to gain such an understanding, to verify if a system behaves as expected, and to detect and diagnose runtime failures. Software monitoring (e.g., logging and tracing) is the key to ensure the observability of large-scale software systems. Despite the importance of software monitoring, several challenges complicate the monitoring of large-scale software systems. First, large-scale software systems are usually composed of many components (e.g., microservices) that may be developed by different organizations using different programming languages. Furthermore, these components are often evolving. The heterogeneous and evolving nature makes it challenging to ensure consistent and up-to-date monitoring of these systems. Second, as the scale and complexity of software increases, so do the requirements on the computing resources needed to produce and manage the monitoring data, which could introduce significant overhead (e.g., performance and storage overhead) to the operations of large-scale software systems. Third, large-scale software systems usually generate a very large amount of monitoring data from multiple sources (e.g., from different web services and applications), which poses challenges for the analysis and utilization of these data. My long-term research goal is to discover, design, and develop a comprehensive and systematic solution to improve software monitoring and increase the observability of large-scale software systems. To achieve this long-term goal, the proposed research program will tackle the challenges of software monitoring from three mutually complementary perspectives: software development, software execution, and analysis of monitoring data. First, we aim to improve the quality of software monitoring by systematically considering the monitoring aspect across the entire software development lifecycle. Second, we aim to increase the cost-effectiveness of software monitoring by dynamically optimizing the monitoring intensity during software execution. Third, we will develop generic approaches to simplify common workflows of analyzing monitoring data. The proposed research program will be a pioneering attempt to systematically improve the monitoring and observability of large-scale software systems, which will benefit the research and practices in Canada in producing high-quality software.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2021-03900
Name nom
Li, Heng
Department département
Department of Computer Engineering and Software Engineering
Organization id
64
Institution établissement
École Polytechnique de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2021
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2706
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Software engineering
Research subject fr
Génie logiciel
Installment
inst:1
Partie
1
Nb partie
5
Application title
Towards Systematic and Cost-Effective Monitoring of Large-Scale Software Systems
Keyword
NSERC
Application summary
Failures of large-scale software systems can have significant impacts on our lives and safety, as well as on the economy and security of our society. For example, failures of the Canada Revenue Agency (CRA) website in March 2017 and March 2019 blocked many Canadians from filing their taxes for days. To detect and address such failures quickly and reduce their impact on users, it is crucial to gain an understanding of the internal state and runtime behavior of these systems. Software observability (i.e., the extent to which the internal state of a system can be inferred) allows software engineers to gain such an understanding, to verify if a system behaves as expected, and to detect and diagnose runtime failures. Software monitoring (e.g., logging and tracing) is the key to ensure the observability of large-scale software systems. Despite the importance of software monitoring, several challenges complicate the monitoring of large-scale software systems. First, large-scale software systems are usually composed of many components (e.g., microservices) that may be developed by different organizations using different programming languages. Furthermore, these components are often evolving. The heterogeneous and evolving nature makes it challenging to ensure consistent and up-to-date monitoring of these systems. Second, as the scale and complexity of software increases, so do the requirements on the computing resources needed to produce and manage the monitoring data, which could introduce significant overhead (e.g., performance and storage overhead) to the operations of large-scale software systems. Third, large-scale software systems usually generate a very large amount of monitoring data from multiple sources (e.g., from different web services and applications), which poses challenges for the analysis and utilization of these data. My long-term research goal is to discover, design, and develop a comprehensive and systematic solution to improve software monitoring and increase the observability of large-scale software systems. To achieve this long-term goal, the proposed research program will tackle the challenges of software monitoring from three mutually complementary perspectives: software development, software execution, and analysis of monitoring data. First, we aim to improve the quality of software monitoring by systematically considering the monitoring aspect across the entire software development lifecycle. Second, we aim to increase the cost-effectiveness of software monitoring by dynamically optimizing the monitoring intensity during software execution. Third, we will develop generic approaches to simplify common workflows of analyzing monitoring data. The proposed research program will be a pioneering attempt to systematically improve the monitoring and observability of large-scale software systems, which will benefit the research and practices in Canada in producing high-quality software.