Grant: Discovery Launch Supplement

Natural Language Processing (NLP) has witnessed significant advancements driven by the emergence of foundational models called large language models (LLMs). These LLMs have showcased remarkable performance in downstream NLP tasks, by serving as a fundamental building block augmented by additional training data and techniques such as pre-training and fine-tuning. These downstream NLP tasks can be broken down into text classification (i.e. sentiment analysis or topic classification) and text generation (i.e. question answering or summarization, or translation). However, the prevailing state-of-the-art LLMs and downstream NLP tasks predominantly revolves around a handful  languages, effectively overlooking the immense potential and significance of over 7000 low-resource languages (LRLs). These LRLs face challenges due to data scarcity, limited computational resources, insufficient language toolkits, and lack of community support (both scientific and technological). To address this gap and need, my research will focus on adapting LLMs to LRLs while driving the progress of their downstream NLP tasks and applications. Research Objective 1. While limited LRL text may exist for a single-language or religious translations, high-quality data for specific tasks and domains often does not exist. To address this limitation, the first objective is to create domain-specific multilingual datasets for LRLs, building upon my prior collaborative efforts.  Research Objective 2. Due to computational and data resource limitations, it is imperative to predict the LRL performance of NLP tasks without the need for training the model. Our previous work has identified factors that influence the performance of LRL using LLMs; therefore, we will improve this work by delving into the factors and statistically formalizing their significance.  Research Objective 3. Building upon our foundational survey in LRL, our research aims to scale additional LRLs using existing massively multilingual pre-trained LLMs.  This research continues my work in the implementation of adaptors with pivot languages, and exploration of exiting methods. In conclusion, this research will pave the way for advancements for LRLs by proposing a groundbreaking techniques that addresses the challenges. By leveraging the latest LLM advancements with cross-lingual transfer, we aim to improve the performance of LRLs downstream tasks based our research in critical factors.  My work will empower numerous trainees with high-demand specialized training in NLP and machine learning, fostering a new generation of researchers. Through this innovation, our research strives to demarginalize and elevate the status of LRLs in the rapidly progressing landscape of LLMs and NLP, thus establishing Canada as the multicultural centre of next generation language technology that is robust to a diversity languages.