Grant: Discovery Grants Program - Individual
University of Toronto
$48,000.00
Resource optimization for in-memory caches
EF 2024-25
Résumé
In-memory caches are DRAM-based caches that are layers between application services and backend storage services. They are ubiquitous in cloud and datacenter environments, where they play a crucial role in improving the performance of data accesses. Two examples of such caches are Memcached and Redis, which are offered as a service by all major cloud service providers. The relevance of providing in-memory caches is that they offload backend storage services for increased scalability, and they reduce query response times significantly by serving data out of memory instead of from the slower backend storage services.
The amount of DRAM memory allocated to each cache impacts its effectiveness: too little leads to increased cache miss ratios and poorer performance, while too much leads to unnecessary capital and operational costs. Surprisingly, most operators over-provision their in-memory caches; e.g., dedicating a separate host with 256GiB/512GiB of memory to each cache instance. The primary reason for this over-provisioning is that the tools to properly size in-memory caches are limited and inadequate. The end result is that a huge amount of DRAM is unnecessarily wasted. Given that large cloud environments (e.g., at Google, Facebook, Amazon, Alibaba, Twitter) use Petabytes of DRAM for in-memory caches, being able to properly size their in-memory caches would save tens of millions a year. But even for smaller organizations, properly sizing their in-memory caches is important given that a single 100GiB in-memory cache at Microsoft Azure costs $100,000 a year. A second reason operators tend to over-provision in-memory caches is legacy: historically, their modus operandi has been to throw hardware at problems that might otherwise demand sophisticated software solutions. It is clear this trend is coming to an end: the unrelenting growth of memory needs of emerging cloud and datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense that can no longer be ignored.
We propose to develop new tools with the objective of optimizing resource (i.e., memory) usage. We will do this in the context of a cache management system whose purpose is to automate the dynamic (re-)configuration of in-memory caches. Our initial studies indicate we can dramatically reduce memory usage without negatively affecting performance, which would lead to substantial cost savings. We further intend to explore the use of disaggregated memory to provide more elasticity. Finally, we will explore the use of specialized in-memory caches for specific but important application classes.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2024-04572
Name nom
Stumm, Michael
Department département
Electrical and Computer Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
48000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2705
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Software and development
Research subject fr
Logiciels et développement
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Resource optimization for in-memory caches
Keywords
computer systems; cloud operations; datacenter operations; caches; storage systems; performance; scalability
Application summary
In-memory caches are DRAM-based caches that are layers between application services and backend storage services. They are ubiquitous in cloud and datacenter environments, where they play a crucial role in improving the performance of data accesses. Two examples of such caches are Memcached and Redis, which are offered as a service by all major cloud service providers. The relevance of providing in-memory caches is that they offload backend storage services for increased scalability, and they reduce query response times significantly by serving data out of memory instead of from the slower backend storage services.
The amount of DRAM memory allocated to each cache impacts its effectiveness: too little leads to increased cache miss ratios and poorer performance, while too much leads to unnecessary capital and operational costs. Surprisingly, most operators over-provision their in-memory caches; e.g., dedicating a separate host with 256GiB/512GiB of memory to each cache instance. The primary reason for this over-provisioning is that the tools to properly size in-memory caches are limited and inadequate. The end result is that a huge amount of DRAM is unnecessarily wasted. Given that large cloud environments (e.g., at Google, Facebook, Amazon, Alibaba, Twitter) use Petabytes of DRAM for in-memory caches, being able to properly size their in-memory caches would save tens of millions a year. But even for smaller organizations, properly sizing their in-memory caches is important given that a single 100GiB in-memory cache at Microsoft Azure costs $100,000 a year. A second reason operators tend to over-provision in-memory caches is legacy: historically, their modus operandi has been to throw hardware at problems that might otherwise demand sophisticated software solutions. It is clear this trend is coming to an end: the unrelenting growth of memory needs of emerging cloud and datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense that can no longer be ignored.
We propose to develop new tools with the objective of optimizing resource (i.e., memory) usage. We will do this in the context of a cache management system whose purpose is to automate the dynamic (re-)configuration of in-memory caches. Our initial studies indicate we can dramatically reduce memory usage without negatively affecting performance, which would lead to substantial cost savings. We further intend to explore the use of disaggregated memory to provide more elasticity. Finally, we will explore the use of specialized in-memory caches for specific but important application classes.