Grant: Discovery Grants Program - Individual
University of Toronto
$91,000.00
Exploiting and Enhancing Programmable Logic for Deep Learning and Datacenter Acceleration
FY 2022-23
Summary
Computing has transformed society, enabling ubiquitous communications, on-demand entertainment, speech recognition, and more. While computation demand is growing for deep learning (DL) and other applications, the traditional efficiency gains afforded by transistor scaling are slowing down. To fill this gap, we need computational devices that are more efficient yet reprogrammable to allow new applications. Field-Programmable Gate Arrays (FPGAs) can be reprogrammed at the hardware level, enabling energy-efficiency gains of 10x or more vs. processors for many embedded and datacenter applications. We seek to advance on three related fronts: implementing efficient DL inference on FPGAs, architecting better reconfigurable devices and enhancing computer-aided design (CAD) tools to enable these new devices. Our first research thrust seeks more efficient DL inference on FPGAs while simultaneously creating productive development flows. In our heterogeneous pipeline (HPIPE) project, we leverage FPGA programmability by implementing customized hardware for every layer in a convolutional neural network (CNN) using a new domain specific compiler. Our neural processing unit (NPU) project instead creates DL functional units controlled by an instruction stream produced from software. Both projects have industry-leading performance, and we will enhance them in multiple ways, including scaling to multiple chips in parallel, exploiting the new tensor blocks in AI-optimized FPGAs, and combining the specialized units of HPIPE with the software programmability of the NPU. Our second thrust seeks new reconfigurable accelerator device (RAD) architectures to allow higher performance and easier development, particularly for DL and for datacenter infrastructure. We will investigate not only conventional 2D chips, but also the multi-die stacks enabled by recent technologies. We envision RADs that combine an FPGA fabric die on an infrastructure die containing coarse-grain programmable accelerators (such as hardened matrix-vector multiply units), large memory blocks, and an embedded network-on-chip (NoC) to link all the components. The combination of FPGA fabric and coarse-grained accelerators can increase performance, while the NoC decouples design components to simplify design. Our third thrust develops the computer-aided design (CAD) tools to investigate these RAD architectures and allow implementation of DL applications on them. First, we will develop a new tool (RADSim) to evaluate fabric, accelerator and NoC combinations by determining execution time for various applications on each architecture. Next, we will enhance the widely-used Versatile Place and Route (VPR) tool to co-optimize the placement of fabric resources, accelerator blocks and NoC routers, with latency and congestion estimates informed by RADsim. The open-source VPR tool is already enabling a wide variety of innovation and products, and these enhancements will make it still more capable.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2022-04445
Name nom
Betz, Vaughn
Department département
Electrical & Computer Engineering - Electrical & Computer Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2022
Award amount
91000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
513
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Electrical and electronic machinery and equipment (including computer hardware)
Area of application fr
Machinerie et équipement électriques et électroniques (y compris matériel informatique)
Research subject code
2500
Research subject group en
Electrical and electronic engineering
Research subject group fr
Génie électrique et électronique
Research subject en
Electrical and electronic engineering
Research subject fr
Génie électrique et électronique
Installment
inst:1
Partie
1
Nb partie
5
Application title
Exploiting and Enhancing Programmable Logic for Deep Learning and Datacenter Acceleration
Application summary
Computing has transformed society, enabling ubiquitous communications, on-demand entertainment, speech recognition, and more. While computation demand is growing for deep learning (DL) and other applications, the traditional efficiency gains afforded by transistor scaling are slowing down. To fill this gap, we need computational devices that are more efficient yet reprogrammable to allow new applications. Field-Programmable Gate Arrays (FPGAs) can be reprogrammed at the hardware level, enabling energy-efficiency gains of 10x or more vs. processors for many embedded and datacenter applications. We seek to advance on three related fronts: implementing efficient DL inference on FPGAs, architecting better reconfigurable devices and enhancing computer-aided design (CAD) tools to enable these new devices. Our first research thrust seeks more efficient DL inference on FPGAs while simultaneously creating productive development flows. In our heterogeneous pipeline (HPIPE) project, we leverage FPGA programmability by implementing customized hardware for every layer in a convolutional neural network (CNN) using a new domain specific compiler. Our neural processing unit (NPU) project instead creates DL functional units controlled by an instruction stream produced from software. Both projects have industry-leading performance, and we will enhance them in multiple ways, including scaling to multiple chips in parallel, exploiting the new tensor blocks in AI-optimized FPGAs, and combining the specialized units of HPIPE with the software programmability of the NPU. Our second thrust seeks new reconfigurable accelerator device (RAD) architectures to allow higher performance and easier development, particularly for DL and for datacenter infrastructure. We will investigate not only conventional 2D chips, but also the multi-die stacks enabled by recent technologies. We envision RADs that combine an FPGA fabric die on an infrastructure die containing coarse-grain programmable accelerators (such as hardened matrix-vector multiply units), large memory blocks, and an embedded network-on-chip (NoC) to link all the components. The combination of FPGA fabric and coarse-grained accelerators can increase performance, while the NoC decouples design components to simplify design. Our third thrust develops the computer-aided design (CAD) tools to investigate these RAD architectures and allow implementation of DL applications on them. First, we will develop a new tool (RADSim) to evaluate fabric, accelerator and NoC combinations by determining execution time for various applications on each architecture. Next, we will enhance the widely-used Versatile Place and Route (VPR) tool to co-optimize the placement of fabric resources, accelerator blocks and NoC routers, with latency and congestion estimates informed by RADsim. The open-source VPR tool is already enabling a wide variety of innovation and products, and these enhancements will make it still more capable.