Grant: Discovery Grants Program - Individual

Large-scale software systems are becoming increasingly dynamic and highly distributed as a result of the emergence of microservice architectures. This architecture is widely used in a variety of industries, including public safety, healthcare, finance, telecommunications, and e-commerce. Despite their many benefits, these systems pose significant operational challenges, making troubleshooting and root cause analysis of faults extremely challenging. Distributed tracing has recently emerged as a promising method that aims to provide full observability into the execution of large-scale microservice systems. It has shown to be useful for diagnosing a range of functional and performance issues in production systems. Industry leaders such as Google, Netflix, and Amazon have popularized this tracing technique in the past few years. While many companies are rushing to implement distributed tracing, the effectiveness of this method in troubleshooting and root cause analysis of faults is still unclear. Recent research studies have revealed important research gaps including the lack of systematic approaches for distributed tracing, lack of intelligent distributed trace analytics tools, challenges associated with high volumes of data, among others. The goal of this research program is to fill this void by developing techniques and tools to improve observability through distributed tracing of large-scale software systems in order to support practitioners when performing fault diagnosis and root cause analysis tasks. Over the next five years, we will focus on three priority research objectives. In the first objective, we will examine how practitioners use distributed tracing in the field to diagnose faults and perform root causes analysis in order to create better distributed tracing analysis tools, as well as to synthesize good and poor practices. The second objective focuses on the challenging problem of identifying automatically parts of the system to trace in response to a system failure. The third objective addresses the challenge of working with large distributed traces, which is a significant impediment to viable trace analysis. To achieve these objectives, we will follow sound research methodologies, while incorporating equity, diversity, and inclusion into all stages of the research program. The outcomes of this research program are expected to have a real impact on the software industry. Improving observability through distributed tracing of large-scale software systems will result in superior failure diagnosis techniques while reducing the associated costs. Another important impact of this research is its potential for commercialization driven by the strong demand from industry to have better observability tools to improve the reliability of complex systems. Furthermore, the proposed program will train a diverse group of HQPs, directly contributing to the Canadian industry, which is in high demand for software specialists across the board.