Advanced Seminar on Law and Technology: Large Language Models Are Unreliable Judges
Date: April 2, 2025 (Wednesday)
Time: 1pm – 2pm
Venue: Academic Conference Room, 11/F Cheng Yu Tung Tower, The University of Hong Kong
Speaker: Jonathan Choi, Professor of Law, University of Southern California Gould School of Law
Can large language models (LLMs) serve as “AI judges” that provide consistent answers to legal questions? I conduct the first series of empirical experiments to test several fundamental questions about LLM reliability for legal interpretation. My findings reveal significant concerns. First, LLM judgments exhibit extreme sensitivity to minor variations in prompt phrasing, giving opposite responses to semantically equivalent questions. Second, LLM judgments are highly sensitive to the model chosen, with virtually no correlation (r = 0.051) between the judgments of different LLMs on identical interpretive questions, suggesting the choice of model could dramatically influence legal outcomes. Third, post-training procedures used to create the most popular models, like ChatGPT and Claude, can cause LLM assessments to substantially deviate from mere predictions of language use in their corpora, which casts doubts on claims that LLMs can elucidate ordinary meaning.
These results challenge the assertion that LLMs provide objective interpretive guidance and suggest they may be vulnerable to strategic manipulation. While LLMs may still have value as interpretive tools, their current limitations demand significant caution from judges and scholars.
Jonathan H. Choi is a professor of law at USC Gould School of Law. He specializes in law and artificial intelligence (applying natural language processing to study legal issues), tax law and statutory interpretation. His work has appeared in the New York University Law Review, the Stanford Law Review, the Yale Journal on Regulation and the Yale Law Journal, among others. His work has been covered by a wide variety of news outlets, including ABC News, Bloomberg, CBS News, CNN, the Daily Mail, Fox News, NBC Nightly News, the New Yorker, Reuters, the Star Tribune, and the Washington Post.
Choi graduated summa cum laude from Dartmouth College, with a triple major in computer science, economics and philosophy and earned high honors for his computer science thesis. He earned a JD at the Yale Law School, where he was the executive bluebook editor of the Yale Law Journal and a founding co-director of the Yale Journal on Regulation Online. Before entering academia, he practiced tax law at Wachtell, Lipton, Rosen & Katz in New York. He previously taught at the University of Minnesota Law School.
Moderator: Benjamin Chen, Associate Professor & Director of the Law and Technology Centre, The University of Hong Kong Faculty of Law
To register, please go to https://hkuems1.hku.hk/hkuems/ec_regform.aspx?guest=Y&UEID=99374. A paper will be circulated in advance and attendees will be expected to have read the paper before the seminar.
For inquiries, please contact Ms. Grace Chan at mcgrace@hku.hk / 3917 4727.