Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Toronto

$65,000.00

Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications

FY 2019-20

Summary
Both big data analytics and mobile cloud applications are increasingly geographically distributed, relying upon services from multiple datacenters around the world. Yet, wide-area inter-datacenter links are costly to build and to upgrade, and inter-datacenter bandwidth is bound to become a scarce resource in the near future. Traditional data analytics need to first move all raw data to one datacenter before they can be processed, which adversely affect their performance and are bandwidth intensive. Real-time streaming analytics and interactive mobile cloud applications such as video conferencing and messaging, on the other hand, require low latencies. As such, there is a pressing need to manage and allocate inter-datacenter resources at a fine granularity to meet the diverse performance targets from both data analytics and mobile cloud applications.****In this program, we propose to design and implement a new software-defined inter-datacenter infrastructure, in order to optimize the utilization of inter-datacenter bandwidth. Operating in the application layer, our infrastructure provides a substrate and a public application programming interface (API) to support existing data analytic frameworks and interactive mobile cloud applications. It schedules data analytic tasks, routes data streams, and manages bandwidth allocations to application flows at a fine granularity, with the objective of utilizing scarce inter-datacenter bandwidth with optimal efficiency. Following the principles of software-defined networking, our infrastructure fully decouples the control and data planes, and makes task scheduling, flow routing, and traffic engineering decisions based on live analytics of network conditions.****We will propose new algorithms to make these decisions in the controller, catering for the specific needs of each category of applications. For batch and streaming analytics applications, our new algorithms are designed to optimize task scheduling and assignment in multiple datacenters, so that both the volume and latency of inter-datacenter traffic can be optimized without prior knowledge of the communication patterns of data analytics jobs. For interactive mobile cloud applications that are bandwidth-intensive and latency-sensitive, our algorithms are designed to optimize resource allocations according to the delay sensitivity of competing application flows. To be more expressive, our infrastructure uses executable scripts to implement control plane decisions in the data plane. We plan to implement our new infrastructure in a real-world inter-datacenter testbed involving multiple datacenters worldwide. This program will lead to fundamentally new ways of using a large number of datacenters around the world to support big data analytics and mobile applications, both of which will soon to be critical and indispensable to businesses and consumers alike in Canada.**
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2016-06281
Name nom
Li, Baochun
Department département
Electrical & Computer Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2016
Award amount
65000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2721
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Multimedia systems and networks
Research subject fr
Systèmes et réseaux multimédias
Installment
inst:4 - 5
Partie
4
Nb partie
5
Application title
Managing Resources across Geo-Distributed Datacenters for Big Data Analytics and Mobile Cloud Applications
Keyword
NSERC
Application summary
Both big data analytics and mobile cloud applications are increasingly geographically distributed, relying upon services from multiple datacenters around the world. Yet, wide-area inter-datacenter links are costly to build and to upgrade, and inter-datacenter bandwidth is bound to become a scarce resource in the near future. Traditional data analytics need to first move all raw data to one datacenter before they can be processed, which adversely affect their performance and are bandwidth intensive. Real-time streaming analytics and interactive mobile cloud applications such as video conferencing and messaging, on the other hand, require low latencies. As such, there is a pressing need to manage and allocate inter-datacenter resources at a fine granularity to meet the diverse performance targets from both data analytics and mobile cloud applications.****In this program, we propose to design and implement a new software-defined inter-datacenter infrastructure, in order to optimize the utilization of inter-datacenter bandwidth. Operating in the application layer, our infrastructure provides a substrate and a public application programming interface (API) to support existing data analytic frameworks and interactive mobile cloud applications. It schedules data analytic tasks, routes data streams, and manages bandwidth allocations to application flows at a fine granularity, with the objective of utilizing scarce inter-datacenter bandwidth with optimal efficiency. Following the principles of software-defined networking, our infrastructure fully decouples the control and data planes, and makes task scheduling, flow routing, and traffic engineering decisions based on live analytics of network conditions.****We will propose new algorithms to make these decisions in the controller, catering for the specific needs of each category of applications. For batch and streaming analytics applications, our new algorithms are designed to optimize task scheduling and assignment in multiple datacenters, so that both the volume and latency of inter-datacenter traffic can be optimized without prior knowledge of the communication patterns of data analytics jobs. For interactive mobile cloud applications that are bandwidth-intensive and latency-sensitive, our algorithms are designed to optimize resource allocations according to the delay sensitivity of competing application flows. To be more expressive, our infrastructure uses executable scripts to implement control plane decisions in the data plane. We plan to implement our new infrastructure in a real-world inter-datacenter testbed involving multiple datacenters worldwide. This program will lead to fundamentally new ways of using a large number of datacenters around the world to support big data analytics and mobile applications, both of which will soon to be critical and indispensable to businesses and consumers alike in Canada.**