Grant: Alliance Grants
University of Calgary
$50,000.00
Learning deep neural network architectures for novel data domains
FY 2023-24
Summary
Deep Neural Networks (DNNs) are the technology behind many of the contemporary Artificial Intelligence (AI) solutions allowing computers to understand photos (Computer Vision), smart assistants to understand our spoken questions (Speech), and translate a foreign language (Natural Language Processing). Much of the success of DNNs in these applications can be attributed to the data domain-specific DNN architectures which allow DNNs to more easily learn compact and meaningful representations within specific data domains. These architectures allow DNNs to be more data efficient, i.e. require less training data to learn to solve a problem, and also increase generalization, i.e. the ability of these representations to perform well for previously unseen data. Examples of such DNN architectures include such as Convolutional Neural Networks (CNNs) for natural images or Transformers for natural language. These architectures were hand-designed using years of research and development to incorporate domain-specific knowledge in the architecture (i.e. structure) of neural network models. Unfortunately applying DNNs to problems within data domains which are poorly addressed by current DNN architectures is fundamentally limited. This includes much of the data collected in industrial informatics, specifically in the agricultural, finance and energy industries. Creating better DNN architectures for these data domains, and problems within these data domains, is expensive, requiring years of research and significant domain and machine learning expertise. We propose instead to explore the potential for state-of-the-art methods of training sparse neural networks to learn DNN structure from training data, specifically in poorly addressed data domains.
Program
Alliance Grants
Recipient
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
577350-2022
Name nom
Ioannou, Yani YA
Department département
Electrical and Software Engineering
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
50000
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
800
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information and communication services
Area of application fr
Services d'information et de télécommunications
Research subject code
2605
Research subject group en
Robotics
Research subject group fr
Pattern analysis and machine intelligence
Research subject en
Robotique
Research subject fr
Analyse de formes et intelligence artificielle
Application title
Learning deep neural network architectures for novel data domains
Keyword
Data Efficient Machine Learning; Deep Neural Networks; Dynamic Neural Networks; Efficient Machine Learning; Inductive Bias; Industrial Informatics; Machine Learning; Neural Network Pruning; Sparse Neural Networks; Structure
Application summary
Deep Neural Networks (DNNs) are the technology behind many of the contemporary Artificial Intelligence (AI) solutions allowing computers to understand photos (Computer Vision), smart assistants to understand our spoken questions (Speech), and translate a foreign language (Natural Language Processing). Much of the success of DNNs in these applications can be attributed to the data domain-specific DNN architectures which allow DNNs to more easily learn compact and meaningful representations within specific data domains. These architectures allow DNNs to be more data efficient, i.e. require less training data to learn to solve a problem, and also increase generalization, i.e. the ability of these representations to perform well for previously unseen data. Examples of such DNN architectures include such as Convolutional Neural Networks (CNNs) for natural images or Transformers for natural language. These architectures were hand-designed using years of research and development to incorporate domain-specific knowledge in the architecture (i.e. structure) of neural network models. Unfortunately applying DNNs to problems within data domains which are poorly addressed by current DNN architectures is fundamentally limited. This includes much of the data collected in industrial informatics, specifically in the agricultural, finance and energy industries. Creating better DNN architectures for these data domains, and problems within these data domains, is expensive, requiring years of research and significant domain and machine learning expertise. We propose instead to explore the potential for state-of-the-art methods of training sparse neural networks to learn DNN structure from training data, specifically in poorly addressed data domains.