Grant: Alliance Grants
York University
$33,317.00
Data Partitioning via Deep Reinforcement Learning
FY 2024-25
Summary
In a partitioned database environment, tables in a database can be located in one or more database partitions. Aside from making data storage more scalable, a partitioned database makes it possible to execute queries over multiple partitions in parallel, thus providing significant performance advantages. However, different ways of partitioning the database may have drastically different implications on query processing performance. For example, co-partitioning two large tables that are frequently joined in the query workload may help vastly reduce the communication cost incurred by data transfer over the network. However, finding the optimal partitioning of a database is non-trivial. Most database management systems (DBMS) today rely on users to specify which and how tables are partitioned. Despite recent advances in the database community that aim to automate the physical database design, a viable solution to the database partitioning is still elusive. Through this project, we propose to utilize deep reinforcement learning to help solve this problem. We aim to provide faster and better partitioning decisions in replace of the current partition advisor for DBMS in a Massively Parallel Processing (MPP) setting, relieving the database administrators from making such decisions manually.
Program
Alliance Grants
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
571801-2021
Name nom
Yu, Xiaohui X
Department département
Information Technology, School of
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2021
Award amount
33317
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2711
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Database management
Research subject fr
Gestion de bases de données
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Data Partitioning via Deep Reinforcement Learning
Keywords
database management systems; data partitioning; physical database design; reinforcement learning
Application summary
In a partitioned database environment, tables in a database can be located in one or more database partitions. Aside from making data storage more scalable, a partitioned database makes it possible to execute queries over multiple partitions in parallel, thus providing significant performance advantages. However, different ways of partitioning the database may have drastically different implications on query processing performance. For example, co-partitioning two large tables that are frequently joined in the query workload may help vastly reduce the communication cost incurred by data transfer over the network. However, finding the optimal partitioning of a database is non-trivial. Most database management systems (DBMS) today rely on users to specify which and how tables are partitioned. Despite recent advances in the database community that aim to automate the physical database design, a viable solution to the database partitioning is still elusive. Through this project, we propose to utilize deep reinforcement learning to help solve this problem. We aim to provide faster and better partitioning decisions in replace of the current partition advisor for DBMS in a Massively Parallel Processing (MPP) setting, relieving the database administrators from making such decisions manually.