Grant: Engage Grants Program
University of British Columbia
$17,262.00
Efficient storage and retrieval techniques for time-indexed data in the cloud
EF 2011-12
Résumé
Gathering high frequency status updates concerning the health of compute servers is a crucial service for
system administrators as more of these servers are migrated to ""the cloud"" (to platforms that are hosted by
third-party providers). Status updates include many metrics including CPU utilization, memory consumption
and disk performance. System administrators require methods to retrieve and visualize such metrics in a timely
manner. Whereas visualization tools are important, they form only one part of any service that provides such
tools. The second part is the reliable, and efficient, management of the data that is gathered from the servers by
the visualization service. Data gathered from the servers operated in the cloud need high durability (potentially
no limit on the archival duration), as well as rapid access when a system administrator seeks data for some
timeframe. For example, a system administrator may need to examine the pattern of arriving requests on a
webserver from the previous few years holiday period when provisioning the webserver for the current holiday
period. Such data should be retrieved by the visualization service within tight timing constraints to provide the
system administrator with meaningful information and with satisfactory performance. Some of the challenges
that would need to be overcome to enable such applications are:
- Scalability: It is important to be able to retrieve large volumes of data with short response times.
- Data availability: Although third-party services employ replication for ensuring high availability in the
presence of failures, recent incidents (such as failure in Amazon's offering) suggest that subscribers may need
to employ alternative strategies as well.
- Bandwidth costs: Third-party hosts charge end-users for the amount of data transferred to and from their
storage services. These costs are not insignificant as the quantity of data being managed can be of the order of
many hundred gigabytes.
Programme
Engage Grants Program
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
417912-2011
Name nom
Gopalakrishnan, Sathish
Department département
Electrical and Computer Engineering
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2011
Competition year année de concours
2011
Award amount
17262
Program id
EGP
Program nane en
Engage Grants Program
Program name fr
Programme de subventions d'engagement partenariat
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1554
Committee name en
Pacific Internal Decision Committee
Committee name fr
Comité de décision interne pour le Pacifique
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2720
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Computer systems software
Research subject fr
Logiciel de systèmes informatiques
Partie
1
Nb partie
1
Application title
Efficient storage and retrieval techniques for time-indexed data in the cloud
Keyword
NSERC
Application summary
Gathering high frequency status updates concerning the health of compute servers is a crucial service for
system administrators as more of these servers are migrated to ""the cloud"" (to platforms that are hosted by
third-party providers). Status updates include many metrics including CPU utilization, memory consumption
and disk performance. System administrators require methods to retrieve and visualize such metrics in a timely
manner. Whereas visualization tools are important, they form only one part of any service that provides such
tools. The second part is the reliable, and efficient, management of the data that is gathered from the servers by
the visualization service. Data gathered from the servers operated in the cloud need high durability (potentially
no limit on the archival duration), as well as rapid access when a system administrator seeks data for some
timeframe. For example, a system administrator may need to examine the pattern of arriving requests on a
webserver from the previous few years holiday period when provisioning the webserver for the current holiday
period. Such data should be retrieved by the visualization service within tight timing constraints to provide the
system administrator with meaningful information and with satisfactory performance. Some of the challenges
that would need to be overcome to enable such applications are:
- Scalability: It is important to be able to retrieve large volumes of data with short response times.
- Data availability: Although third-party services employ replication for ensuring high availability in the
presence of failures, recent incidents (such as failure in Amazon's offering) suggest that subscribers may need
to employ alternative strategies as well.
- Bandwidth costs: Third-party hosts charge end-users for the amount of data transferred to and from their
storage services. These costs are not insignificant as the quantity of data being managed can be of the order of
many hundred gigabytes.