Grant: Discovery Grants Program - Individual
University of British Columbia
$18,000.00
Parallel and Distributed Computing for Real-time Analytics
FY 2018-19
Summary
The goal of my research is to create a runtime system that can scale-out to large size clusters of multicore machines for in-memory computation of analytics on big data. Many of the existing distributed systems do not take advantage of the tools and techniques developed in high performance computing for addressing the performance issues that ultimately limits scalability. MPI (Message-Passing Interface) is a message-passing library that is the de-facto standard in parallel programming for high performance computing. I intend to develop new MPI middleware along with associated tools as the basis for a scalable runtime system for computing analytics.*Scaling-out requires the ability to express massive amounts of concurrency and ultimately relies on message-passing. As such, the starting point for this work will be our FG-MPI (Fine-Grain MPI (Message-Passing Interface)) system, which we have shown can scale to thousands and millions of processes. FG-MPI extends MPICH, an MPI runtime system that is used to derive most of the commercial versions of MPI. This work will require extending the existing FG-MPI middleware as well as introducing new tools and techniques for programming at large scale.*The research involves (a) investigating further improvements to the MPI middleware to support multicore machines, (b) developing tools to support an ""actor-like"" programming environment for specifying the composition of processes and the development of service-oriented data structures for MPI, (c) following a million process methodology to develop techniques for developing and deploying programs at scale, (d) developing innovative message-based data structures and algorithms to support knowledge-based analysis and reasoning on big data. *This research directly addresses the two main trends in computer technology: the appearance of multicore and cluster/cloud computing. In both cases --- the future is parallel! The central problem is how to use multicore to scale-up or how to use more machines to scale-out to provide the performance needed to process and compute analytics on the vast amount of data that is available today.*
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2015-04934
Name nom
Wagner, Alan
Department département
Computer Science
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2018
Competition year année de concours
2015
Award amount
18000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2704
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Distributed and parallel processing
Research subject fr
Traitement réparti et simultané
Partie
4
Nb partie
5
Application title
Parallel and Distributed Computing for Real-time Analytics
Keyword
NSERC
Application summary
The goal of my research is to create a runtime system that can scale-out to large size clusters of multicore machines for in-memory computation of analytics on big data. Many of the existing distributed systems do not take advantage of the tools and techniques developed in high performance computing for addressing the performance issues that ultimately limits scalability. MPI (Message-Passing Interface) is a message-passing library that is the de-facto standard in parallel programming for high performance computing. I intend to develop new MPI middleware along with associated tools as the basis for a scalable runtime system for computing analytics.*Scaling-out requires the ability to express massive amounts of concurrency and ultimately relies on message-passing. As such, the starting point for this work will be our FG-MPI (Fine-Grain MPI (Message-Passing Interface)) system, which we have shown can scale to thousands and millions of processes. FG-MPI extends MPICH, an MPI runtime system that is used to derive most of the commercial versions of MPI. This work will require extending the existing FG-MPI middleware as well as introducing new tools and techniques for programming at large scale.*The research involves (a) investigating further improvements to the MPI middleware to support multicore machines, (b) developing tools to support an ""actor-like"" programming environment for specifying the composition of processes and the development of service-oriented data structures for MPI, (c) following a million process methodology to develop techniques for developing and deploying programs at scale, (d) developing innovative message-based data structures and algorithms to support knowledge-based analysis and reasoning on big data. *This research directly addresses the two main trends in computer technology: the appearance of multicore and cluster/cloud computing. In both cases --- the future is parallel! The central problem is how to use multicore to scale-up or how to use more machines to scale-out to provide the performance needed to process and compute analytics on the vast amount of data that is available today.*