Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Toronto

$82,000.00

Storage and Memory Management for Data-Intensive Computing

FY 2022-23

Summary
Computation is increasingly moving into data centres with the widespread adoption of the Cloud computing model, while data centre computing itself is being radically transformed by the availability of very large main memories, the commodification of new hardware technologies such as Infiniband networking with Remote Direct Memory Access (RDMA) capabilities, and high capacity solid state disk (SSD) storage. In this environment, a rack of compute nodes has become the new data centre building block, known as rack-scale computing. Also, non-volatile memory is becoming a commercial reality with the introduction of Intel's 3D XPoint technology. This hardware revolution has profound implications for systems-level software, especially the storage and memory management layers, as old performance bottlenecks are eliminated and new ones emerge. Hardware advances and distributed data processing frameworks are fueling rapid advances in data analytics and machine learning, enabling applications in automated language translation, image understanding and autonomous vehicles (to name just a few). Today's frameworks are often scalable but highly inefficient, squandering the potential of new hardware with excessive software overheads. My research explores ways to manage the burgeoning wealth of hardware resources, enabling future data-intensive analytics applications to operate at larger scales, more efficiently, and with greater reliability. We summarize three specific avenues to achieve this goal.1. A unified rack-scale persistent memory storage system: Existing research on persistent memory storage improves on traditional file systems designed for slow, block-based devices, but it does not consider issues of allocation, access control, and consistency when scaling beyond a single node. Future rack-scale computing will demand a scalable, parallel and distributed file system that exposes the performance of emerging persistent memory across a full rack of nodes. 2. Memory-locality aware scheduling for data analytics frameworks: Many large-scale data analytics applications use Apache Spark or similar computing frameworks, which are designed with the assumption that data is stored on slow-to-access disks. Disk locality is of decreasing importance when nodes in a rack are connected by high-speed networks. We are exploring ways to exploit information about what parts of the input data are already in memory at each node, making smarter task placement possible. 3. In-memory distributed spatiotemporal data analytics: The rise of sensors and GPS-equipped mobile devices is driving rapid growth of spatiotemporal data, enabling novel applications that require sophisticated spatiotemporal data analytics. The expanding data volume means these datasets may soon exceed the capacity of even large-memory servers, however, requiring new techniques for in-memory rack-scale processing.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2018-06391
Name nom
Demke-Brown, Angela
Department département
Computer Science (St. George Campus) - Computer Science (St. George Campus)
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2018
Award amount
82000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2700
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Information technology
Research subject fr
Technologie de l'information
Installment
inst:5;6
Partie
2
Nb partie
6
Application title
Storage and Memory Management for Data-Intensive Computing
Application summary
Computation is increasingly moving into data centres with the widespread adoption of the Cloud computing model, while data centre computing itself is being radically transformed by the availability of very large main memories, the commodification of new hardware technologies such as Infiniband networking with Remote Direct Memory Access (RDMA) capabilities, and high capacity solid state disk (SSD) storage. In this environment, a rack of compute nodes has become the new data centre building block, known as rack-scale computing. Also, non-volatile memory is becoming a commercial reality with the introduction of Intel's 3D XPoint technology. This hardware revolution has profound implications for systems-level software, especially the storage and memory management layers, as old performance bottlenecks are eliminated and new ones emerge. Hardware advances and distributed data processing frameworks are fueling rapid advances in data analytics and machine learning, enabling applications in automated language translation, image understanding and autonomous vehicles (to name just a few). Today's frameworks are often scalable but highly inefficient, squandering the potential of new hardware with excessive software overheads. My research explores ways to manage the burgeoning wealth of hardware resources, enabling future data-intensive analytics applications to operate at larger scales, more efficiently, and with greater reliability. We summarize three specific avenues to achieve this goal.1. A unified rack-scale persistent memory storage system: Existing research on persistent memory storage improves on traditional file systems designed for slow, block-based devices, but it does not consider issues of allocation, access control, and consistency when scaling beyond a single node. Future rack-scale computing will demand a scalable, parallel and distributed file system that exposes the performance of emerging persistent memory across a full rack of nodes. 2. Memory-locality aware scheduling for data analytics frameworks: Many large-scale data analytics applications use Apache Spark or similar computing frameworks, which are designed with the assumption that data is stored on slow-to-access disks. Disk locality is of decreasing importance when nodes in a rack are connected by high-speed networks. We are exploring ways to exploit information about what parts of the input data are already in memory at each node, making smarter task placement possible. 3. In-memory distributed spatiotemporal data analytics: The rise of sensors and GPS-equipped mobile devices is driving rapid growth of spatiotemporal data, enabling novel applications that require sophisticated spatiotemporal data analytics. The expanding data volume means these datasets may soon exceed the capacity of even large-memory servers, however, requiring new techniques for in-memory rack-scale processing.