Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Alberta

$42,000.00

Reliability and Explainability of Large Language Models

FY 2024-25

Summary
Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP), achieving state-of-the-art performance on most tasks, sometimes without any training. Question-answering systems based on LLMs like ChatGPT are rapidly replacing Web search engines for non-transactional searches and are widely used in professional and educational settings. Motivated by the potential impact of LLMs in education, this research program aims to answer the question: can an LLM be trusted on a specific topic or coding task? (e.g., explaining loop unrolling or rewriting an SQL query). As pointed out by OpenAI, there is a genuine danger with these systems: the better they get, the more users trust them despite their many known flaws. Even the most capable LLM to date, GPT-4, is prone to "hallucinate" due to problems with training data and the inherently generative nature of these models. Because LLMs perform best with in-distribution inputs, detecting Out-Of- Distribution (OOD) inputs could help answer our question, but only under the assumptions that the LLM is trained (only) on correct information and that one has access to the training data; both are problematic. Today, LLMs are evaluated primarily through benchmarking. In fact, millions of users flock to LLMs because of flashy results on benchmarks related to coding, question answering, or other tasks. However, ad-hoc benchmarks can be misleading because they reduce extremely complex system behaviour into a single, uninterpretable number. Also, benchmarks are biased towards popular tasks and questions, typically found online, which are themselves used in LLM training: It is no surprise that LLMs perform well on them. Nevertheless, they can fail on long-tail questions and tasks and most users have no way of knowing that. We offer a sound and more nuanced alternative to evaluate LLMs that builds on the PI's proven expertise and established research on computer-based evaluation of knowledge and the use of AI in education. We will systematically and exhaustively evaluate LLMs of different characteristics and sizes using potentially millions of questions and coding tasks generated to cover a domain of interest. We will help users determine which topics or tasks their favourite LLM will likely provide (in)correct answers. We will push the state-of-the-art in Automatic Question Generation, Information Extraction, and Natural Language Understanding, which remain active and vibrant research areas. All resources and corpora needed for this research already exist. Finally, we will train HQP in highly desirable skills needed to work with LLMs at a large scale. Our methods are LLM-agnostic and will work with future LLMs or QA systems based on other technologies. Also, while this program focuses on challenging computational problems related to our objectives, it will contribute tools and resources that can be used to improve open LLMs and develop educational tools to help human learners.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2024-04941
Name nom
Barbosa, Denilson
Department département
Computing Science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
42000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2801
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Natural language and speech understanding
Research subject fr
Langage naturel et reconnaissance de la parole
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Reliability and Explainability of Large Language Models
Keywords
large language models; natural language processing; robustness; reliability; explainability; systematic assessment of knowledge; automatic question generation
Application summary
Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP), achieving state-of-the-art performance on most tasks, sometimes without any training. Question-answering systems based on LLMs like ChatGPT are rapidly replacing Web search engines for non-transactional searches and are widely used in professional and educational settings. Motivated by the potential impact of LLMs in education, this research program aims to answer the question: can an LLM be trusted on a specific topic or coding task? (e.g., explaining loop unrolling or rewriting an SQL query). As pointed out by OpenAI, there is a genuine danger with these systems: the better they get, the more users trust them despite their many known flaws. Even the most capable LLM to date, GPT-4, is prone to "hallucinate" due to problems with training data and the inherently generative nature of these models. Because LLMs perform best with in-distribution inputs, detecting Out-Of- Distribution (OOD) inputs could help answer our question, but only under the assumptions that the LLM is trained (only) on correct information and that one has access to the training data; both are problematic. Today, LLMs are evaluated primarily through benchmarking. In fact, millions of users flock to LLMs because of flashy results on benchmarks related to coding, question answering, or other tasks. However, ad-hoc benchmarks can be misleading because they reduce extremely complex system behaviour into a single, uninterpretable number. Also, benchmarks are biased towards popular tasks and questions, typically found online, which are themselves used in LLM training: It is no surprise that LLMs perform well on them. Nevertheless, they can fail on long-tail questions and tasks and most users have no way of knowing that. We offer a sound and more nuanced alternative to evaluate LLMs that builds on the PI's proven expertise and established research on computer-based evaluation of knowledge and the use of AI in education. We will systematically and exhaustively evaluate LLMs of different characteristics and sizes using potentially millions of questions and coding tasks generated to cover a domain of interest. We will help users determine which topics or tasks their favourite LLM will likely provide (in)correct answers. We will push the state-of-the-art in Automatic Question Generation, Information Extraction, and Natural Language Understanding, which remain active and vibrant research areas. All resources and corpora needed for this research already exist. Finally, we will train HQP in highly desirable skills needed to work with LLMs at a large scale. Our methods are LLM-agnostic and will work with future LLMs or QA systems based on other technologies. Also, while this program focuses on challenging computational problems related to our objectives, it will contribute tools and resources that can be used to improve open LLMs and develop educational tools to help human learners.