Grant: Alliance Grants

The written knowledge of human experience is immense and rich. However, much of this knowledge is unbeknownst and inaccessible. Natural language processing (NLP) represents one avenue to unlock this valuable information by understanding text automatically. This potential is being realized with the recent emergence of foundation or large language models that combine the power of self-supervised deep neural networks with unfathomably large and broad data sets. One hallmark of these models is the concept of "transfer learning," whereby the model is trained to do one task and can be fine-tuned to complete a related but different task. In this proposal, the team will focus on recent advances with large language models (e.g., BERT, GPT-3) in NLP to extract valuable information from unstructured documents. Despite the lack of cohesion and organization in unstructured documents, they often come with embedded structured data. Representing useful and structured information as tables is a common and effective method in scientific, financial, health and many domains for information retrieval and other tasks. However, manual extraction of structured data from documents typically costs tremendous time and labour, motivating the need for a system for automating the process. Specifically, this proposal will leverage groundbreaking advances by BERT and T5 models to develop tools for automated table extraction from documents. That is, given a collection of "fields of interest" (FoI), the task is to automatically extract and populate a table with the FoIs as column headers. To illustrate that our tools are general with broad applicability, we demonstrate the effectiveness of our algorithms with two very different application domains: (1) breast cancer care; and (2) mineral mining. After such tables have been extracted, the data can be used for various "downstream" analytics tasks such as question answering and predictive modeling.