Grant: Collaborative Research and Development Grants

As organizations grow, their data centres inevitably become larger and more complex. These centres usually provide numerous services to the internal and external users; such services are commonly deployed on virtual machines (VMs) across a cluster of computing devices, which can either be the property of the organization (i.e., a ""private cloud"") or be rented from a public cloud provider such as Amazon EC2. In both cases, private or public, managing ""the cloud"" requires appropriate virtualization and cost-effective provisioning of the infrastructure with respect to the type and size of the service requests (i.e., the workload). The implementation of this virtualization and provisioning needs to be carefully considered. Over-estimating the required infrastructure for a set of requested services in a specified period of time leads to waste, under-utilization, and increased costs. At the same time, under-estimation of the future workload is also unacceptable, since it eventually degrades the quality of the services and may lead to violations of in the Service-Level Agreements (SLAs) with the clients. Minimizing the infrastructure costs, while at the same time ensuring that SLAs are satisfied, requires not only state-of-the-art techniques for resource assignment and load balancing but also a detailed understanding of the cloud workload itself.This research project, called DOPPLER, will study approaches for a) understanding and modelling the cloud workloads by mining the relevant ""big data"" in the cloud to characterize the workloads, and b) optimizing cloud provisioning and management by providing advanced workload forecasts for the cloud. Short-term forecasting, requiring accurate prediction of the coming workload based on the recent history and the observed trend and seasonality, will be investigated for dynamic placement and provisioning. In addition, longer-term predictions will be evaluated for modelling longer-term trends, which in turn can be used for planning and provisioning of future resources. Finally, the modelling and forecasting approaches will be empirically evaluated in the context of the cloud environments of our industry partners: CA Technologies and its clients.