Grant: Discovery Grants Program - Individual
The University of British Columbia
$52,000.00
Improving and Leveraging Discourse Processing in Modern NLP
FY 2023-24
Summary
Research on Natural Language Processing (NLP) has recently made rapid advancements thanks to breakthroughs in machine learning, with extremely large deep learning language models dominating the field. However, despite their amazing success, these models still struggle when interpreting and generating long text, often missing parts of its discourse structure, namely how the sentences and paragraphs in the text are related to each other in order to express complex ideas. The main goal of the proposed research is to address this critical limitation, so that modern NLP systems can be broadly and safely deployed to benefit society by performing extremely useful tasks like text summarization, question answering, machine translation and interactive dialog. Technically, endowing neural language models with more sophisticated discourse knowledge presents many difficult challenges. First, to automatically extend the size and domain coverage of annotated training data, we will devise novel techniques for distant supervision and for exploiting generic autoencoding objectives. As these larger datasets become available, we will leverage them in processing discourse. The challenge for discourse parsing will be integrating existing approaches in more accurate and interpretable solutions, while for discourse generation, it will be to develop sophisticated methods for improving the contet and coherence of the generated text, by injecting useful semantic, discourse and other pragmatic cues in the generation process. Throughout the project, we will test our solutions mainly on text summarization, an extremely useful NLP downstream task that can provide users with efficient access to huge repositories of textual data. Here, we will need to deal with the complexities of leveraging discourse information in the encoder-decoder architecture at the core of modern neural summarizers. In addition to scientific impact, we expect this research to have real and significant socioeconomic benefits for Canada. Tellingly, AI is a priority area for the federal government and NLP, the focus of this proposal, is a thriving subfield of AI. Furthermore, the specific software solutions resulting from my program will be very valuable to the rapidly increasing number of Canadian and foreign companies developing NLP tools. Users of these tools range from journalists producing news, to citizens making sense of them; from doctors examining patient histories and relevant medical literature, to patients exploring on-line discussions about their condition. The proposed research has great potential to fuel these and many other applications, by boosting the performance of deep learning models with sophisticated discourse abilities, and by turning these improvements into robust and versatile text processing technologies.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2023-05174
Name nom
Carenini, Giuseppe
Department département
computer science
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2023
Award amount
52000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
800
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information and communication services
Area of application fr
Services d'information et de télécommunications
Research subject code
2801
Research subject group en
Artificial intelligence
Research subject group fr
Natural language and speech understanding
Research subject en
Intelligence artificielle
Research subject fr
Langage naturel et reconnaissance de la parole
Application title
Improving and Leveraging Discourse Processing in Modern NLP
Keyword
autoencoders; discourse parsing and planning; discourse processing; discourse relations; distant supervision; language models; multi-lingual; natural language processing; pragmatics; text summarization
Application summary
Research on Natural Language Processing (NLP) has recently made rapid advancements thanks to breakthroughs in machine learning, with extremely large deep learning language models dominating the field. However, despite their amazing success, these models still struggle when interpreting and generating long text, often missing parts of its discourse structure, namely how the sentences and paragraphs in the text are related to each other in order to express complex ideas. The main goal of the proposed research is to address this critical limitation, so that modern NLP systems can be broadly and safely deployed to benefit society by performing extremely useful tasks like text summarization, question answering, machine translation and interactive dialog. Technically, endowing neural language models with more sophisticated discourse knowledge presents many difficult challenges. First, to automatically extend the size and domain coverage of annotated training data, we will devise novel techniques for distant supervision and for exploiting generic autoencoding objectives. As these larger datasets become available, we will leverage them in processing discourse. The challenge for discourse parsing will be integrating existing approaches in more accurate and interpretable solutions, while for discourse generation, it will be to develop sophisticated methods for improving the contet and coherence of the generated text, by injecting useful semantic, discourse and other pragmatic cues in the generation process. Throughout the project, we will test our solutions mainly on text summarization, an extremely useful NLP downstream task that can provide users with efficient access to huge repositories of textual data. Here, we will need to deal with the complexities of leveraging discourse information in the encoder-decoder architecture at the core of modern neural summarizers. In addition to scientific impact, we expect this research to have real and significant socioeconomic benefits for Canada. Tellingly, AI is a priority area for the federal government and NLP, the focus of this proposal, is a thriving subfield of AI. Furthermore, the specific software solutions resulting from my program will be very valuable to the rapidly increasing number of Canadian and foreign companies developing NLP tools. Users of these tools range from journalists producing news, to citizens making sense of them; from doctors examining patient histories and relevant medical literature, to patients exploring on-line discussions about their condition. The proposed research has great potential to fuel these and many other applications, by boosting the performance of deep learning models with sophisticated discourse abilities, and by turning these improvements into robust and versatile text processing technologies.