Grant: Discovery Grants Program - Individual
University of Toronto
$59,000.00
The Data Centre as a Heterogeneous CPU
FY 2024-25
Summary
Most people today are familiar with the term "the cloud", which is where magic happens to provide services like social media, streaming movies, and the applications accessed using our mobile devices. What powers the cloud are large data centres that are football-field sized facilities housing upwards of more than a hundred thousand compute servers. These servers are high-performance microprocesor-based systems (CPUs) that can be programmed to execute any application. There are two primary concerns when operating these data centres: power and performance. The largest data centres consume over 100 megawatts of power with the corresponding impact on carbon emissions. Performance is always a criteria because we want faster response times.
To address the power and performance challenge, data centres have started to incorporate specialized computing devices that are more efficient in power and better at performance, but only for specific types of computation, such as for machine learning. These devices are typically called "accelerators". By introducing accelerators, the data centre has evolved from a homogeneous collection of CPUs to a heterogeneous collection of many different types of computing devices.
The goal of this proposal is to address the challenge of programming applications to efficiently and easily leverage the heterogeneous resources in a data centre. Our approach is to view a data centre collectively as a programmable entity. A simple description is to say that in a way similar to how we write a software program that can be run on a single CPU, we wish to write a program that can be run in a data centre. Essentially, we treat the data centre as a CPU and we are building a programming environment that will target this "Data Centre CPU". The inspiration comes from examining how a modern CPU is constructed. A modern CPU is a collection of processing elements, such as adders, multipliers, and other functional units. The software to program the CPU is converted into instructions that sequence data through the processing elements. A heterogeneous data centre can be considered to be a collection of processing elements that are programmed to implement tasks, and we seek to develop a way to describe a large application as a program, discover the tasks required and then sequence data through the tasks to execute the application. Exemplary examples of applications that can leverage such a platform are the large-scale transformer networks now being used for natural language processing.
A successful realization of this vision will lead to a new level of computing that can make it feasible to build ever larger applications that require significant heterogeneous computing power, while doing it efficiently with lower power. Computing is now the backbone of almost everything we do and ways to efficiently harness the ever growing computing resources available will be a benefit to all Canadians and everyone in the world.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2023-05284
Name nom
Chow, Paul
Department département
Electrical and Computer Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
59000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2719
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Computer architecture and design
Research subject fr
Architecture d'ordinateur et conception
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
The Data Centre as a Heterogeneous CPU
Keywords
heterogeneous computing systems; reconfigurable computing; parallel and distributed computing; field programmable gate arrays; computer systems; compilers; hardware description languages; operating systems; programming languages; hardware acceleration
Application summary
Most people today are familiar with the term "the cloud", which is where magic happens to provide services like social media, streaming movies, and the applications accessed using our mobile devices. What powers the cloud are large data centres that are football-field sized facilities housing upwards of more than a hundred thousand compute servers. These servers are high-performance microprocesor-based systems (CPUs) that can be programmed to execute any application. There are two primary concerns when operating these data centres: power and performance. The largest data centres consume over 100 megawatts of power with the corresponding impact on carbon emissions. Performance is always a criteria because we want faster response times.
To address the power and performance challenge, data centres have started to incorporate specialized computing devices that are more efficient in power and better at performance, but only for specific types of computation, such as for machine learning. These devices are typically called "accelerators". By introducing accelerators, the data centre has evolved from a homogeneous collection of CPUs to a heterogeneous collection of many different types of computing devices.
The goal of this proposal is to address the challenge of programming applications to efficiently and easily leverage the heterogeneous resources in a data centre. Our approach is to view a data centre collectively as a programmable entity. A simple description is to say that in a way similar to how we write a software program that can be run on a single CPU, we wish to write a program that can be run in a data centre. Essentially, we treat the data centre as a CPU and we are building a programming environment that will target this "Data Centre CPU". The inspiration comes from examining how a modern CPU is constructed. A modern CPU is a collection of processing elements, such as adders, multipliers, and other functional units. The software to program the CPU is converted into instructions that sequence data through the processing elements. A heterogeneous data centre can be considered to be a collection of processing elements that are programmed to implement tasks, and we seek to develop a way to describe a large application as a program, discover the tasks required and then sequence data through the tasks to execute the application. Exemplary examples of applications that can leverage such a platform are the large-scale transformer networks now being used for natural language processing.
A successful realization of this vision will lead to a new level of computing that can make it feasible to build ever larger applications that require significant heterogeneous computing power, while doing it efficiently with lower power. Computing is now the backbone of almost everything we do and ways to efficiently harness the ever growing computing resources available will be a benefit to all Canadians and everyone in the world.