Grant: DRA: Canada Impact+ Research Training Awards - Doctoral

"Artificial intelligence is transforming how people access mental health support, offering accessible conversations to those in need. However, current Large Language Models (LLMs) suffer from a critical safety flaw known as ""sycophancy."" This occurs when an LLM tries so hard to be helpful and agreeable that it unintentionally validates a user's harmful thoughts, delusions, or incorrect medical beliefs. In a clinical context, an AI that prioritizes ""being nice"" over ""being truthful"" can reinforce dangerous behaviors or delay professional care. This research project proposes a comprehensive solution to make mental health AI safer and more trustworthy. The work is divided into three actionable phases. First, we will survey young adults to understand precisely where the line is between supportive empathy and unsafe compliance. We need to know what users perceive as helpful versus what medical professionals deem safe. Second, we will use these insights to build a rigorous testing system. This ""benchmark"" will stress-test AI models with complex, emotional scenarios to measure how often they fall into the trap of unsafe agreement. Finally, we will develop and test new engineering safeguards. This involves retraining models to distinguish between validating a user's feelings (which is good) and agreeing with false facts (which is bad). We will also prototype a ""safety-by-design"" architecture that checks AI responses for risks before they reach the user. Ultimately, this research aims to set a new standard for digital health. By solving the sycophancy problem, we can provide Canadian health systems with a blueprint for deploying AI tools that are not only technologically advanced but also fundamentally safe, empathetic, and trustworthy for vulnerable populations."