Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

McGill University

$78,000.00

Towards Robust Unsupervised Language Learning

EF 2023-24

Résumé
You know many things about new words, even before you have ever heard them used. For instance, you know that a new messaging app is more likely to be called a nonsense word like “jimble” than one like “jinble”. You know that if it is called “jimble,” then someone might say “jimble me a message,” but probably wouldn't say “jimble me with a message.” You know that a person who uses the app could be called a “jimbler,” and many other facts about how this word can and can't be used.Understanding the computational processes that underlie this kind of generalization is a central problem for language science and is crucial to engineering computer systems capable of interacting with people. Despite tremendous progress in recent years, there remains a large gap between human-level natural language learning and the state-of-the-art in natural language processing. In particular, state-of-the-art systems tend to require supervision on specific tasks and large quantities of training data. This contrasts with human learners who acquire flexible knowledge like that above, in a largely unsupervised fashion. My research focuses on building systems that generalize in more human-like ways by incorporating theoretical insights from linguistics into probabilistic models of unsupervised language learning.This project has two objectives. First, I will use techniques from probabilistic programming to develop systems for specifying models of unsupervised language learning and for automatically deriving inference algorithms from such specifications. This work will build on insights from mathematical linguistics as well as recent advances in probabilistic inference. In particular, I will make use of results about the formal complexity of linguistic theories as well as tools from “variational Bayesian” inference. Second, I will make use of these frameworks to explore a variety of models that encode substantive theoretical assumptions from the linguistics literature about the sound structure of words and the structure of sentences.This project will have several important impacts. On the engineering side, it will advance the state-of-the-art in unsupervised language learning. By discovering the most effective representations and proposals from linguistic theory, we will move towards the goal of systems that can generalize robustly from small amounts of data. The inference tools developed in the project will also make it possible to rapidly integrate future insights from linguistic theory. On the scientific side, this work will allow for rapid prototyping and testing of assumptions present in different linguistic theories and thus provide a powerful new tool for testing theoretical assumptions in order to advance our understanding of the linguistic system.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2018-06385
Name nom
O'Donnell, Timothy
Department département
Linguistics
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2018
Award amount
78000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Towards Robust Unsupervised Language Learning
Keyword
Computational Cognitive Science; Computational Linguistics; Computational Phonetics and Phonology; Grammar Induction; Low Resource Languages; Natural Language Processing; Probabilistic Modeling; Probabilistic Programming; Speech Processing; Unsupervised Language Learning
Application summary
You know many things about new words, even before you have ever heard them used. For instance, you know that a new messaging app is more likely to be called a nonsense word like “jimble” than one like “jinble”. You know that if it is called “jimble,” then someone might say “jimble me a message,” but probably wouldn't say “jimble me with a message.” You know that a person who uses the app could be called a “jimbler,” and many other facts about how this word can and can't be used.Understanding the computational processes that underlie this kind of generalization is a central problem for language science and is crucial to engineering computer systems capable of interacting with people. Despite tremendous progress in recent years, there remains a large gap between human-level natural language learning and the state-of-the-art in natural language processing. In particular, state-of-the-art systems tend to require supervision on specific tasks and large quantities of training data. This contrasts with human learners who acquire flexible knowledge like that above, in a largely unsupervised fashion. My research focuses on building systems that generalize in more human-like ways by incorporating theoretical insights from linguistics into probabilistic models of unsupervised language learning.This project has two objectives. First, I will use techniques from probabilistic programming to develop systems for specifying models of unsupervised language learning and for automatically deriving inference algorithms from such specifications. This work will build on insights from mathematical linguistics as well as recent advances in probabilistic inference. In particular, I will make use of results about the formal complexity of linguistic theories as well as tools from “variational Bayesian” inference. Second, I will make use of these frameworks to explore a variety of models that encode substantive theoretical assumptions from the linguistics literature about the sound structure of words and the structure of sentences.This project will have several important impacts. On the engineering side, it will advance the state-of-the-art in unsupervised language learning. By discovering the most effective representations and proposals from linguistic theory, we will move towards the goal of systems that can generalize robustly from small amounts of data. The inference tools developed in the project will also make it possible to rapidly integrate future insights from linguistic theory. On the scientific side, this work will allow for rapid prototyping and testing of assumptions present in different linguistic theories and thus provide a powerful new tool for testing theoretical assumptions in order to advance our understanding of the linguistic system.