Grant: Discovery Grants Program - Individual
University of Waterloo
$41,000.00
Neural generative models for natural language
FY 2021-22
Summary
Neural networks are powerful machine learning methods that have pushed the boundaries of research in Natural Language Processing (NLP) and led to the development of many successful applications. The proposed research is aimed at developing neural network models for controlled natural language generation. Neural networks are in many cases black boxes, as it is often impossible to interpret the internal (latent) representations of the input learned by the models. One way to control the characteristics of the generated text is to learn the representations that are interpretable by humans. For that we need to disentangle the learned latent representations of different factors of variations of the input text. In our recent research, my students and I developed a neural model that successfully disentangles the representation of style (sentiment) and content of text, and demonstrated its effectiveness in text style transfer. Controllable text generation is a fast developing research field, and there is an urgency to develop neural models to enable many practical applications, including dialogue systems, text style transfer and summarization. Our first objective is to develop models for multi-class style transfer. Examples of style categories are author's style or emotions expressed in text. Practical applications of style transfer include writing assistants that automatically suggest alternate versions of user-written text that adhere to the target style better. Another application of such models is dialogue systems, which generate responses conditioned on a specific style, persona or emotion. Our second objective is to develop neural models that disentangle the latent representations of syntactic and semantic information in text. The premise is that the same idea can be expressed in many different ways syntactically. This disentanglement will potentially allow us to perform more complex style transfer by controlling the structure of generated sentences. Another important implication of such models is a potentially better representation learning of text meaning compared to the current models, which encode both syntactic and semantic characteristics in the same latent space. This will have practical benefits for many downstream tasks, such as paraphrase detection, which is essential for natural language understanding. The third objective is to learn long-term features that are characteristic of longer text sequences, such as paragraphs and entire documents, and to disentangle their latent representation from the sentence-level representations. This research has several potential implications, such as maintaining stylistic consistency of generated sentences throughout the document, and generating coherent multi-sentence sequences. The proposed research will provide training opportunities for six graduate students, who will acquire state-of-the-art knowledge and practical skills in deep learning, which is one of the top emerging areas of employment.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2019-04897
Name nom
Vechtomova, Olga
Department département
Management Sciences
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2019
Award amount
41000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Installment
inst:3
Partie
1
Nb partie
5
Application title
Neural generative models for natural language
Keyword
NSERC
Application summary
Neural networks are powerful machine learning methods that have pushed the boundaries of research in Natural Language Processing (NLP) and led to the development of many successful applications. The proposed research is aimed at developing neural network models for controlled natural language generation. Neural networks are in many cases black boxes, as it is often impossible to interpret the internal (latent) representations of the input learned by the models. One way to control the characteristics of the generated text is to learn the representations that are interpretable by humans. For that we need to disentangle the learned latent representations of different factors of variations of the input text. In our recent research, my students and I developed a neural model that successfully disentangles the representation of style (sentiment) and content of text, and demonstrated its effectiveness in text style transfer. Controllable text generation is a fast developing research field, and there is an urgency to develop neural models to enable many practical applications, including dialogue systems, text style transfer and summarization. Our first objective is to develop models for multi-class style transfer. Examples of style categories are author's style or emotions expressed in text. Practical applications of style transfer include writing assistants that automatically suggest alternate versions of user-written text that adhere to the target style better. Another application of such models is dialogue systems, which generate responses conditioned on a specific style, persona or emotion. Our second objective is to develop neural models that disentangle the latent representations of syntactic and semantic information in text. The premise is that the same idea can be expressed in many different ways syntactically. This disentanglement will potentially allow us to perform more complex style transfer by controlling the structure of generated sentences. Another important implication of such models is a potentially better representation learning of text meaning compared to the current models, which encode both syntactic and semantic characteristics in the same latent space. This will have practical benefits for many downstream tasks, such as paraphrase detection, which is essential for natural language understanding. The third objective is to learn long-term features that are characteristic of longer text sequences, such as paragraphs and entire documents, and to disentangle their latent representation from the sentence-level representations. This research has several potential implications, such as maintaining stylistic consistency of generated sentences throughout the document, and generating coherent multi-sentence sequences. The proposed research will provide training opportunities for six graduate students, who will acquire state-of-the-art knowledge and practical skills in deep learning, which is one of the top emerging areas of employment.