Grant: Alliance Grants

Canada's legal industry is experiencing significant changes due to the rise of Artificial Intelligence (AI), particularly Large Language Models (LLMs) like ChatGPT. These technologies offer innovative ways to support legal tasks such as document review, research, and court filing, potentially making high-quality legal support more accessible to the public. However, these AI models face challenges that hinder their effectiveness in legal practice. For example, they can generate false information ("hallucination") or make incorrect assumptions ("contra-factual bias"), which could worsen inequalities in access to legal help. There is also a risk that these tools might create a "legal monoculture" by limiting diverse perspectives. Artificial Intelligence must be carefully aligned with domain expertise to ensure the delivery of trustworthy, domain-specific AI systems. While some initiatives have been undertaken in this direction (Sara Merken 2023), most industry solutions appear to be underperforming. A recent landmark study indicates that legal AI solutions developed by leading legal software companies exhibit poor performance, with high hallucination rates comparable to those seen with GPT-4 (Magesh et al. 2024). Our project, led by the Queen's Conflict Analytics Lab (CAL), aims to address these issues. We will develop and test open-source and custom AI solutions through our platform, OpenJustice, which allows for crowdsourced and programmable legal AI development. We have partnered with over 40 organizations, including law schools and legal aid centers, to ensure our models receive high-quality feedback. We will focus on three main areas: 1. Developing Reliable Legal AI Models: We will refine AI models to better handle legal tasks, using feedback from legal professionals. 2. Creating Custom Legal AI Solutions: We will design AI models tailored to specific firms' needs, helping with tasks like negotiation and legal annotations. 3. Establishing Performance Standards: We will compare our AI models with general-purpose ones to evaluate their effectiveness and address issues like misinformation and bias. The expected outcomes include generating valuable evidence on AI effectiveness, potentially creating new AI products, and improving access to legal assistance. Our findings will help inform future regulations and standards for legal AI, ensuring these tools benefit the Canadian legal community as well as Canadians seeking legal help.