Grant: Engage Grants Program

Cloud infrastructure has not been sufficiently developed for innovative Canadian SMEs working with volatile big data and requiring real-time analytic capability. Moreover, SMEs are, by nature, lean and require cost-aware solutions. While cloud infrastructure makes large data computing affordable, SMEs are seeking to further minimize the rents they pay for cloud infrastructure. Cost-aware and customizable auto-scaling solutions for cloud infrastructure are needed. In addition, current large file management systems such as Hadoop are insufficient to support real-time big data analytics. The objectives of this research are to address the scientific challenges to (1) design a data architecture to support real-time big-data analytics on highly volatile data, and (2) investigate, design, and implement a cost-aware, customizable, hybrid cloud infrastructure, and proprietary cloud bursting technology for auto-scaling. The auto-scaler will manage dynamic allocation of resources on Select Technology Corporation's custom cloud which currently consists of a hybrid of CANARIE cloud servers, the company's own private cloud, and other commercial cloud vendors; The volatility of the big data presents unique data management challenges that have not yet been solved in state-of-the-art solutions. Our auto-scaler solution approaches will employ the creation of cost-aware models for allocation of resources across multiple cloud platforms and will customize the auto-scaler's performance for real-time analytic workloads with the use of prior knowledge of a global client workflow model. Our data architecture solutions will involve investigating the performance tradeoffs between novel hybridizations of high-volume transactional database servers and big data file schemes, and pure open source large distributed file storage systems.