Grant: Discovery Grants Program - Individual
University of Waterloo
$62,000.00
Automated Cloud Provisioning and Management
FY 2016-17
Summary
Cloud computing harnesses massive resource capacity in data centers to support service applications in a scalable and cost-effective manner. The successful deployment of Cloud computing infrastructures and technologies in recent years is transforming the way IT business operates. However, the rapid growth in scale and complexity of today's Cloud environments also raises critical concerns regarding the scalability and effectiveness of the underlying management systems, making Cloud provisioning and resource management an increasingly difficult challenge.
To address this challenge, our long term objective is to design a scalable, efficient and reliable Cloud resource management framework that incurs minimal management overhead. Our general approach consists in (1) analyzing workload and resource characteristics, and (2) developing resource management schemes that leverage workload and resource characterizations. Our approach is motivated by the observation that Cloud resources (e.g., physical machines and network equipment) and workload (jobs and applications) often have heterogeneous yet characterizable behavior. For example, user-facing applications often have periodic (e.g., daily and seasonal) demand patterns and short-term trends. Similarly, batch (e.g., MapReduce) jobs with similar configurations are often executed in data centers on a routinely basis. Thus, by leveraging the knowledge gained from workload characterizations, it is possible to devise informed resource management schemes that attain higher application performance and resource utilization while reducing energy consumption. Specifically, the project aims at achieving the following 4 short-term objectives:
(1) Characterizing workload in Cloud data centers is a relatively new research topic, especially for recent big-data applications such as MapReduce and Spark. Our recent work has shown that there is significant heterogeneity of both resources (e.g. capacity, performance, reliability and energy efficiency of physical hardware) and workloads (e.g. arrival rate, priority, resource usage, communication patterns and running time). Currently there is a lack of models for capturing such heterogeneity in a scalable yet accurate fashion. We intend to apply techniques from statistics and machine learning to address this challenge.
(2) Despite extensive research on scheduling, the studies on resource-aware and performance (e.g. deadline)-aware scheduling for big-data analytics are rather limited. In this context, we plan to leverage resource and application profiles to make informed scheduling decisions with consideration to fairness, performance and resource efficiency.
(3) Minimizing energy cost is a major concern of today's Cloud providers. However, existing work on this topic often overlooks the heterogeneity in Cloud data centers. We intend to develop adaptive heterogeneity-aware energy management solutions to balance the trade-off between energy efficiency and application performance.
(4) We also intend to extend our management framework to the context of geo-distributed clouds. In these settings, it is a major challenge to jointly provision service components (e.g., servers, load balancers, caches, proxies) and network resources to meet performance requirements in a dynamic manner. We intend to develop provisioning schemes that consider demand patterns to improve the performance and efficiency of geo-distributed services.
The outcome of this project will significantly advance the state-of-the-art on Cloud resource provisioning and management. It will also have a profound impact on the development of industrial Cloud management systems. Finally, it will provide excellent opportunities for training highly qualified personal at the PhD and Master's levels.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2014-04533
Name nom
Boutaba, Raouf
Department département
Computer Science, David R. Cheriton School of
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2016
Competition year année de concours
2014
Award amount
62000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
804
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Communication systems and services (planning, organization, services)
Area of application fr
Systèmes et services de communications (planification, organisation et services)
Research subject code
2721
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Multimedia systems and networks
Research subject fr
Systèmes et réseaux multimédias
Partie
3
Nb partie
5
Application title
Automated Cloud Provisioning and Management
Keyword
NSERC
Application summary
Cloud computing harnesses massive resource capacity in data centers to support service applications in a scalable and cost-effective manner. The successful deployment of Cloud computing infrastructures and technologies in recent years is transforming the way IT business operates. However, the rapid growth in scale and complexity of today's Cloud environments also raises critical concerns regarding the scalability and effectiveness of the underlying management systems, making Cloud provisioning and resource management an increasingly difficult challenge.
To address this challenge, our long term objective is to design a scalable, efficient and reliable Cloud resource management framework that incurs minimal management overhead. Our general approach consists in (1) analyzing workload and resource characteristics, and (2) developing resource management schemes that leverage workload and resource characterizations. Our approach is motivated by the observation that Cloud resources (e.g., physical machines and network equipment) and workload (jobs and applications) often have heterogeneous yet characterizable behavior. For example, user-facing applications often have periodic (e.g., daily and seasonal) demand patterns and short-term trends. Similarly, batch (e.g., MapReduce) jobs with similar configurations are often executed in data centers on a routinely basis. Thus, by leveraging the knowledge gained from workload characterizations, it is possible to devise informed resource management schemes that attain higher application performance and resource utilization while reducing energy consumption. Specifically, the project aims at achieving the following 4 short-term objectives:
(1) Characterizing workload in Cloud data centers is a relatively new research topic, especially for recent big-data applications such as MapReduce and Spark. Our recent work has shown that there is significant heterogeneity of both resources (e.g. capacity, performance, reliability and energy efficiency of physical hardware) and workloads (e.g. arrival rate, priority, resource usage, communication patterns and running time). Currently there is a lack of models for capturing such heterogeneity in a scalable yet accurate fashion. We intend to apply techniques from statistics and machine learning to address this challenge.
(2) Despite extensive research on scheduling, the studies on resource-aware and performance (e.g. deadline)-aware scheduling for big-data analytics are rather limited. In this context, we plan to leverage resource and application profiles to make informed scheduling decisions with consideration to fairness, performance and resource efficiency.
(3) Minimizing energy cost is a major concern of today's Cloud providers. However, existing work on this topic often overlooks the heterogeneity in Cloud data centers. We intend to develop adaptive heterogeneity-aware energy management solutions to balance the trade-off between energy efficiency and application performance.
(4) We also intend to extend our management framework to the context of geo-distributed clouds. In these settings, it is a major challenge to jointly provision service components (e.g., servers, load balancers, caches, proxies) and network resources to meet performance requirements in a dynamic manner. We intend to develop provisioning schemes that consider demand patterns to improve the performance and efficiency of geo-distributed services.
The outcome of this project will significantly advance the state-of-the-art on Cloud resource provisioning and management. It will also have a profound impact on the development of industrial Cloud management systems. Finally, it will provide excellent opportunities for training highly qualified personal at the PhD and Master's levels.