Grant: Alliance Grants
York University
$48,567.00
Data Partitioning via Deep Reinforcement Learning
FY 2023-24
Summary
In a partitioned database environment, tables in a database can be located in one or more database partitions. Aside from making data storage more scalable, a partitioned database makes it possible to execute queries over multiple partitions in parallel, thus providing significant performance advantages. However, different ways of partitioning the database may have drastically different implications on query processing performance. For example, co-partitioning two large tables that are frequently joined in the query workload may help vastly reduce the communication cost incurred by data transfer over the network. However, finding the optimal partitioning of a database is non-trivial. Most database management systems (DBMS) today rely on users to specify which and how tables are partitioned. Despite recent advances in the database community that aim to automate the physical database design, a viable solution to the database partitioning is still elusive. Through this project, we propose to utilize deep reinforcement learning to help solve this problem. We aim to provide faster and better partitioning decisions in replace of the current partition advisor for DBMS in a Massively Parallel Processing (MPP) setting, relieving the database administrators from making such decisions manually.
Program
Alliance Grants
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
571801-2021
Name nom
Yu, Xiaohui X
Department département
Information Technology, School of
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
48567
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Data Partitioning via Deep Reinforcement Learning
Keyword
database management systems; data partitioning; physical database design; reinforcement learning
Application summary
In a partitioned database environment, tables in a database can be located in one or more database partitions. Aside from making data storage more scalable, a partitioned database makes it possible to execute queries over multiple partitions in parallel, thus providing significant performance advantages. However, different ways of partitioning the database may have drastically different implications on query processing performance. For example, co-partitioning two large tables that are frequently joined in the query workload may help vastly reduce the communication cost incurred by data transfer over the network. However, finding the optimal partitioning of a database is non-trivial. Most database management systems (DBMS) today rely on users to specify which and how tables are partitioned. Despite recent advances in the database community that aim to automate the physical database design, a viable solution to the database partitioning is still elusive. Through this project, we propose to utilize deep reinforcement learning to help solve this problem. We aim to provide faster and better partitioning decisions in replace of the current partition advisor for DBMS in a Massively Parallel Processing (MPP) setting, relieving the database administrators from making such decisions manually.
