Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

Dalhousie University

$26,000.00

Deep learning with limited data

EF 2018-19

Résumé
Deep neural networks is a machine learning technique based on a model of neuron-like elements. The ""deep"" refers here to networks with many layers that are capable of representing complex functions. Such models have been proven to be extremely valuable as they can learn hierarchical data representations from examples and predict previously unseen data. Such techniques enabled much of the recent progress in computer vision and pattern recognition, and the advancements of deep neural networks was made possible by the availability of large data sets (Big Data), some refinement of algorithms, and the availability of increased computational power with specialized processors such as graphical processing units (GPUs). Companies like Google, Microsoft, Amazon and Facebook, and also numerous smaller engineering firms and market analysis firms are embracing this new technology. *However, many applications do not have the luxury of having a large set of examples available to train complex models. In the research program proposed here I am trying to develop methods for applications with limited data, which includes a variety of situations such as small number of measurements, biased examples, or examples from peripheral data. Applications with limited data are common. For example, in the ocean science community it is common to have a fairly small coverage of measurements compared to the vastness of the oceans. My lab has started to investigate how artificial data can augment pre-training and facilitate the narrowing of the search space. For example, there exist a variety of physical models of ocean dynamics such as the exchange of CO2 with the atmosphere. While these models are generally considered to be insufficient for predictions on the required scale, it is possible that these simulations can provide data for the pre-training of deep networks.*Another area that promises a whole new level of techniques to training deep networks is the guidance of the learning process by domain experts, also called transfer learning. Domain experts include hereby either deep neural networks that have been trained on complementary or similar domains, or even human experts. While transfer learning has been considered for some time, the use of deep networks to learn appropriate communication channels for learning opened exciting new possibilities for much progress in this area. Such expert training would provide network training that parallels human learning. Such techniques could further drastically reduce the need of large training sets. *The long-term goal of this research program is to develop specific methods and tools to both evaluate a machine-learning problem in terms of its complexity and data need, and to provide a comprehensive toolbox to apply pre-learning and expert transfer learning strategies to problem domains that suffer from limited data.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Nova Scotia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2017-05117
Name nom
Trappenberg, Thomas
Department département
Computer Science, Faculty of
Organization id
79
Institution établissement
Dalhousie University
Province en
Nova Scotia
Province fr
Nouvelle-Écosse
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2018
Competition year année de concours
2017
Award amount
26000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
518
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Medical equipment and apparatus
Area of application fr
Appareils et instruments médicaux
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
2
Nb partie
5
Application title
Deep learning with limited data
Keyword
NSERC
Application summary
Deep neural networks is a machine learning technique based on a model of neuron-like elements. The ""deep"" refers here to networks with many layers that are capable of representing complex functions. Such models have been proven to be extremely valuable as they can learn hierarchical data representations from examples and predict previously unseen data. Such techniques enabled much of the recent progress in computer vision and pattern recognition, and the advancements of deep neural networks was made possible by the availability of large data sets (Big Data), some refinement of algorithms, and the availability of increased computational power with specialized processors such as graphical processing units (GPUs). Companies like Google, Microsoft, Amazon and Facebook, and also numerous smaller engineering firms and market analysis firms are embracing this new technology. *However, many applications do not have the luxury of having a large set of examples available to train complex models. In the research program proposed here I am trying to develop methods for applications with limited data, which includes a variety of situations such as small number of measurements, biased examples, or examples from peripheral data. Applications with limited data are common. For example, in the ocean science community it is common to have a fairly small coverage of measurements compared to the vastness of the oceans. My lab has started to investigate how artificial data can augment pre-training and facilitate the narrowing of the search space. For example, there exist a variety of physical models of ocean dynamics such as the exchange of CO2 with the atmosphere. While these models are generally considered to be insufficient for predictions on the required scale, it is possible that these simulations can provide data for the pre-training of deep networks.*Another area that promises a whole new level of techniques to training deep networks is the guidance of the learning process by domain experts, also called transfer learning. Domain experts include hereby either deep neural networks that have been trained on complementary or similar domains, or even human experts. While transfer learning has been considered for some time, the use of deep networks to learn appropriate communication channels for learning opened exciting new possibilities for much progress in this area. Such expert training would provide network training that parallels human learning. Such techniques could further drastically reduce the need of large training sets. *The long-term goal of this research program is to develop specific methods and tools to both evaluate a machine-learning problem in terms of its complexity and data need, and to provide a comprehensive toolbox to apply pre-learning and expert transfer learning strategies to problem domains that suffer from limited data.