Grant: Discovery Grants Program - Individual

Our ability to acquire, process, and store huge streams of data has transformed business, culture, technology, and science. In the prevailing cloud computing paradigm, we first move the data to a data center where it can be processed and stored efficiently. Within the data center, stream processing is a popular choice when timely up-to-date responses must be computed from a practically infinite volume of data generated by mobile devices, sensors, users, and infrastructure. However, experts predict over 40 billion devices connected to the internet by the year 2025, such as mobile phones, environmental sensors, and smart devices. The cloud computing paradigm may not keep up with the flood: cloud computing will not help if we cannot get the data into the data center in the first place. Edge computing is an emerging alternative to cloud computing. It uses resources close to where the data originates, as well as on the path to the cloud, to provide computation and storage over a very large number of sites connected by relatively slow and unreliable data links. The potential social and economic benefits of stream processing on the edge are huge, with applications in every aspect of modern life: smart traffic management and smart electricity grids reduce travel time and greenhouse gas emissions; environmental monitoring of water and air quality help control pollution; quick detection and prediction of earthquakes or tsunamis save thousands of lives; and continuous health monitoring with wearable sensors improve quality of life. Yet despite their promise, in practice such systems are rare since they currently call for substantial engineering effort. First, existing software for distributed stream processing was designed to work inside the cloud and works poorly in edge computing settings due to incompatible assumptions. Second, current approaches for distributed data analysis are too limited, or require mathematical sophistication and expertise beyond the reach of typical software developers. The proposed research program will develop a new generation of distributed stream processing infrastructure for emerging edge computing applications: it will be efficient on both edge and cloud, and holistically span the spectrum between them; it will general, rather than limited to a small set of pre-built computations; and it will be accessible to non-experts, enabling wider access to edge computing. The program includes three major directions: (1) develop techniques for automatically distributing any computation, (2) create a stream processing software designed from the ground-up to meet the demands of future edge computing applications, and (3) address questions of performance and reliability. This program will be a game changer for Canadian developers: it will lower the barrier to entry for edge computing, spark innovative products that support diverse communities, and provide the infrastructure to power next generation analytics and machine learning.