Grant: Discovery Grants Program - Individual
Université de Montréal
$70,000.00
Learning deep architectures for AI
FY 2013-14
Summary
Machine learning algorithms attempt to endow machines with the ability to capture knowledge through examples, e.g., allowing a machine to classify or predict correctly in new cases. Machine learning research has been extremely successful in the past two decades and is now applied in many areas of science and technology, a well known example being web search engines. Yet, machines still seem to fall short of even mammal-level intelligence in many respects. One of the remaining frontiers of machine learning is the difficulty of learning the kind of complicated and highly-varying functions that are necessary to perform machine vision or natural language processing tasks at a level comparable to humans. Deep learning methods aim at learning of hierarchies of concepts, with concepts from higher-levels of the hierarchy formed by the composition of lower level ones. Automatically learning features at multiple levels of abstraction allows a system to learn complex functions mapping the input to the output directly from data. This is especially important for higher-level abstractions, which humans often do not know how to specify explicitly. Although theory suggests that deep architectures, which are found in the brain, could be very important for AI, they come with a difficult optimization challenge, which is the subject of the proposed research. It is encouraging to note that animals and humans apply sub-optimal but wholly adequate strategies towards this optimization difficulty, and we will seek inspiration from the natural world in this research. We propose hypotheses to explain this difficulty and the success of the methods recently introduced, along with experiments to help us understand this optimization difficulty, and new algorithms to improve the training of deep architectures on large datasets involving only few human-labeled examples.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
138071-2009
Name nom
Bengio, Yoshua
Department département
Informatique et recherche opérationnelle
Organization id
63
Institution établissement
Université de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2013
Competition year année de concours
2009
Award amount
70000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
331
Committee name en
Computing and Information Sciences - B
Committee name fr
Informatique - B
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
5
Nb partie
5
Application title
Learning deep architectures for AI
Keyword
NSERC
Application summary
Machine learning algorithms attempt to endow machines with the ability to capture knowledge through examples, e.g., allowing a machine to classify or predict correctly in new cases. Machine learning research has been extremely successful in the past two decades and is now applied in many areas of science and technology, a well known example being web search engines. Yet, machines still seem to fall short of even mammal-level intelligence in many respects. One of the remaining frontiers of machine learning is the difficulty of learning the kind of complicated and highly-varying functions that are necessary to perform machine vision or natural language processing tasks at a level comparable to humans. Deep learning methods aim at learning of hierarchies of concepts, with concepts from higher-levels of the hierarchy formed by the composition of lower level ones. Automatically learning features at multiple levels of abstraction allows a system to learn complex functions mapping the input to the output directly from data. This is especially important for higher-level abstractions, which humans often do not know how to specify explicitly. Although theory suggests that deep architectures, which are found in the brain, could be very important for AI, they come with a difficult optimization challenge, which is the subject of the proposed research. It is encouraging to note that animals and humans apply sub-optimal but wholly adequate strategies towards this optimization difficulty, and we will seek inspiration from the natural world in this research. We propose hypotheses to explain this difficulty and the success of the methods recently introduced, along with experiments to help us understand this optimization difficulty, and new algorithms to improve the training of deep architectures on large datasets involving only few human-labeled examples.