Grant: Alliance Grants
York University
$25,000.00
Hallucination Detection in Large Language Model Responses using Metamorphic Testing
EF 2024-25
Résumé
We propose to develop a comprehensive suite of novel techniques and tools to automatically detect hallucinations in the responses of Large Language Models (LLMs), such as ChatGPT. Our techniques will leverage Metamorphic Testing (MT) which is found to be effective to test software systems from diverse domains, including Machine learning (ML) systems. This Catalyst grant will form a team between leading experts in Canada and the UK on ML and software testing, and will form the basis of a long-term research program that will contribute to the development of theory and tools for AI safety assessment and risk mitgation using software testing and verification techniques.
Programme
Alliance Grants
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
597429-2024
Name nom
Uddin, Gias MG
Department département
Electrical Engineering & Computer Science
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
25000
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2804
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Expert systems
Research subject fr
Systèmes experts
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Hallucination Detection in Large Language Model Responses using Metamorphic Testing
Keywords
fairness in ai/ml; large language models; metamorphic testing; software testing
Application summary
We propose to develop a comprehensive suite of novel techniques and tools to automatically detect hallucinations in the responses of Large Language Models (LLMs), such as ChatGPT. Our techniques will leverage Metamorphic Testing (MT) which is found to be effective to test software systems from diverse domains, including Machine learning (ML) systems. This Catalyst grant will form a team between leading experts in Canada and the UK on ML and software testing, and will form the basis of a long-term research program that will contribute to the development of theory and tools for AI safety assessment and risk mitgation using software testing and verification techniques.