Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Master's Award: Canada Graduate Scholarships

University of Toronto

$27,000.00

Evaluating Large Language Models for Chronic Disease Management: A Practical and Scalable Approach to Ensure Long-Term Efficacy and Feasibility

FY 2024-25

Summary
Chronic diseases, which last for 3 months or longer, account for 89% of deaths and over $80 billion in annual healthcare costs in Canada. The long-term nature of these conditions demands ongoing, personalized care. The integration of Large Language Models (LLMs), such as ChatGPT, in healthcare is promising, as LLMs can generate human-like text for summarization, question-answering, and clinical support. However, LLMs also pose risks, such as generating factually incorrect or biased information, highlighting the need for robust evaluation methods. Current evaluations are inadequate, research shows that with only 5% using real patient data and limited focus on metrics like fairness, bias, and empathy.This research aims to evaluate existing LLM evaluation frameworks, particularly in chronic disease management. It seeks to propose a practical, scalable evaluation framework using data science and analytics tools like time-series analysis and visualization to assess LLM applications for long-term disease management. Specifically, this research explores the question of how should we evaluate LLM applications for chronic disease management? What tools in data analytics and science ensure that evaluation methods are feasible, scalable, and relevant for long-term needs?This study aim to addresses the gap in LLM evaluation specific to chronic diseases, providing practical insights on how LLMs can be safely and effectively deployed in healthcare settings. By using real-world data, this research offers applied solutions for improving clinical outcomes, ensuring LLMs meet the needs of chronic disease patients before broader implementation.
Program
Master's Award: Canada Graduate Scholarships
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
CIHR Funding Decisions Database

As published

Id
530189
Name
Liu Xinyi
Region
Ontario
Country
Canada
Orgtype
Institution Paid; Research Institution
Prcname2
Special Cases - Awards Programs
Projecttitle
Evaluating Large Language Models for Chronic Disease Management: A Practical and Scalable Approach to Ensure Long-Term Efficacy and Feasibility
Programtype2
Studentship Programs
Orgnameinp2
University of Toronto
Cihramount2
$27,000
Cihrequipment2
$0
Programname2
Master's Award: Canada Graduate Scholarships
Keyworddelim
Artificial Intelligence; Data Science; Digital Health; Health Service Evaluation; Health System Research; Implementation Science; Large Language Model
Abstract
Chronic diseases, which last for 3 months or longer, account for 89% of deaths and over $80 billion in annual healthcare costs in Canada. The long-term nature of these conditions demands ongoing, personalized care. The integration of Large Language Models (LLMs), such as ChatGPT, in healthcare is promising, as LLMs can generate human-like text for summarization, question-answering, and clinical support. However, LLMs also pose risks, such as generating factually incorrect or biased information, highlighting the need for robust evaluation methods. Current evaluations are inadequate, research shows that with only 5% using real patient data and limited focus on metrics like fairness, bias, and empathy.This research aims to evaluate existing LLM evaluation frameworks, particularly in chronic disease management. It seeks to propose a practical, scalable evaluation framework using data science and analytics tools like time-series analysis and visualization to assess LLM applications for long-term disease management. Specifically, this research explores the question of how should we evaluate LLM applications for chronic disease management? What tools in data analytics and science ensure that evaluation methods are feasible, scalable, and relevant for long-term needs?This study aim to addresses the gap in LLM evaluation specific to chronic diseases, providing practical insights on how LLMs can be safely and effectively deployed in healthcare settings. By using real-world data, this research offers applied solutions for improving clinical outcomes, ensuring LLMs meet the needs of chronic disease patients before broader implementation.
Competitiondate
202412
Approvedterm2
1 yr 0 mth
Pinamesdelim
Liu, Xinyi
Approvedterm
1 an 0 mois
Cihrcontribution2
~ $27,000~ $0~