Grant: Discovery Grants Program - Individual
University of Waterloo
$28,000.00
Automatic Deep Neural Network Design by a Novel Stochastic Search Process
FY 2022-23
Summary
The promising advances made in machine learning and especially deep learning have led to better modeling accuracy for different applications such as image classification, object detection, speech recognition, and even medical applications. However, the design cycle of these models still rely on manual, time-consuming, and very complex process, which requires high-level expertise. Consequently, the design and development of specialized and efficient neural network models using such time-consuming manual processes are very difficult slowing down their adoption in many real-world applications. Such shortcoming becomes even more critical especially when it is necessary to take into account the inherent limitations in the computational power and the memory of the available hardware. Neural architecture search (NAS) methods offer a solution to address these challenges. NAS designs the underlying graph structure of models automatically by heuristically searching the solution space to find the most optimal structure. Despite their promising potentials, NAS approaches are still in their infancy, computationally complex and limited to specific applications. The proposed research program aims to mitigate these challenges. The following objectives will be investigated: i) Investigate stochastic branch prediction strategies to reduce the dimensionality of the search space architecture and accordingly the computational complexity. ii) Characterize the complex information inside the computational graph of a deep learning model based on the new stochastic process. iii) Incorporate hardware characteristics in the design process and develop a novel approach to generate hardware-specific optimized computational graphs for the deep learning models. vi) Design and develop more adaptive machine learning systems via the novel method which take environmental properties into account. The results of the proposed research program will increase the applicability of deep neural networks in different industries such as autonomous driving, consumer electronics and healthcare. Due to its automatic nature, the proposed approach offers dramatically reduced design cycles, which facilitates developing applications based on this technology with much less level of end-user expertise. This new approach will also boost the adoption of deep learning models in different industries in Canada as a pioneer country in the field of AI. The knowledge and technologies developed in the proposed research program will be transferred through active collaborations with industrial partners such as Microsoft, Intel and DarwinAI. This ensures the efficient development and deployment of deep neural networks for real-world Machine Learning applications. HQP trained in this program will become experts in the field of artificial intelligence and machine learning, putting them in a strong position for AI leadership roles in industry as well as in academia, where there is currently a shortage of experts.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-04469
Name nom
Shafiee, Mohammad Javad
Department département
Systems Design Engineering - Systems Design Engineering
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2020
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2600
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Robotics
Research subject fr
Robotique
Installment
inst:3
Partie
1
Nb partie
5
Application title
Automatic Deep Neural Network Design by a Novel Stochastic Search Process
Application summary
The promising advances made in machine learning and especially deep learning have led to better modeling accuracy for different applications such as image classification, object detection, speech recognition, and even medical applications. However, the design cycle of these models still rely on manual, time-consuming, and very complex process, which requires high-level expertise. Consequently, the design and development of specialized and efficient neural network models using such time-consuming manual processes are very difficult slowing down their adoption in many real-world applications. Such shortcoming becomes even more critical especially when it is necessary to take into account the inherent limitations in the computational power and the memory of the available hardware. Neural architecture search (NAS) methods offer a solution to address these challenges. NAS designs the underlying graph structure of models automatically by heuristically searching the solution space to find the most optimal structure. Despite their promising potentials, NAS approaches are still in their infancy, computationally complex and limited to specific applications. The proposed research program aims to mitigate these challenges. The following objectives will be investigated: i) Investigate stochastic branch prediction strategies to reduce the dimensionality of the search space architecture and accordingly the computational complexity. ii) Characterize the complex information inside the computational graph of a deep learning model based on the new stochastic process. iii) Incorporate hardware characteristics in the design process and develop a novel approach to generate hardware-specific optimized computational graphs for the deep learning models. vi) Design and develop more adaptive machine learning systems via the novel method which take environmental properties into account. The results of the proposed research program will increase the applicability of deep neural networks in different industries such as autonomous driving, consumer electronics and healthcare. Due to its automatic nature, the proposed approach offers dramatically reduced design cycles, which facilitates developing applications based on this technology with much less level of end-user expertise. This new approach will also boost the adoption of deep learning models in different industries in Canada as a pioneer country in the field of AI. The knowledge and technologies developed in the proposed research program will be transferred through active collaborations with industrial partners such as Microsoft, Intel and DarwinAI. This ensures the efficient development and deployment of deep neural networks for real-world Machine Learning applications. HQP trained in this program will become experts in the field of artificial intelligence and machine learning, putting them in a strong position for AI leadership roles in industry as well as in academia, where there is currently a shortage of experts.