A new federally supported research partnership between the University of British Columbia and Vancouver-based legaltech startup Caseway aims to tackle one of the most pressing challenges in legal AI: inaccurate and misleading answers generated by large language models.
The two-year project is funded through the NSERC Alliance–Mitacs Accelerate program and will focus on improving the reliability of a legal research engine built on large language models.
The research is led by Vered Shwartz, an assistant professor of computer science at UBC, whose group includes postdoctoral researcher Maksym Taranukhin and master’s student Ethan Zhao.
The initiative comes amid growing concern about AI hallucinations in legal contexts. A study published last year found that when prompted with legal questions, large language models produced answers inconsistent with legal facts at least 58% of the time.
“Large language models are already deployed and people are using them to get answers for legal questions,” said Shwartz. “Our goal is to ensure that these AI tools are as reliable as possible so that professionals and the public can trust them to provide accurate legal information.”
While many AI systems rely on retrieval-augmented generation—allowing models to search for and reference external information—errors remain common. According to Taranukhin, deploying such systems prematurely in legal settings carries real risk. “Rushing AI into legal use without fixes could be harmful, especially in high-stakes contexts where people need to rely on accurate information to make important decisions,” he said.
To address these issues, the research team will integrate advanced reasoning techniques into the system, including Bayesian networks, which model probabilistic relationships between legal factors. The project will also explore ways for the system to recognize uncertainty and refrain from answering when confidence is low—an important safeguard in legal use cases.
One of the core challenges, Shwartz noted, is balancing usability with accuracy. “There’s a tradeoff between how helpful it is and how accurate it is,” she said. “It’s tricky to strike the right balance.”
The project builds on the group’s prior work in legal AI, including earlier research led by Taranukhin on a chatbot designed to educate air passengers about their legal rights.
For Caseway, the collaboration aligns with its mission to expand access to trustworthy legal information. “Getting the right information quickly can make all the difference,” said Alistair Vigier, CEO of Caseway. “We’re not building this to replace lawyers, but to make legal knowledge far more accessible to those who can’t easily get professional help.”
The researchers hope the work will contribute to safer, more dependable legal AI tools—particularly as public and professional reliance on generative AI continues to grow.