Eleventh Circuit concurrence could be a watershed moment for discourse on the judiciary’s future use of AI.
In a very thought-provoking recent ruling, Judge Kevin Newsom of the 11th Circuit Court of Appeals discussed the potential use of AI-powered large language models (LLMs) in legal text interpretation. Known for his commitment to textualism and plain-language interpretation, Judge Newsom’s career – prior to President Trump appointing him to the bench in 2017 – includes serving as the Solicitor General of Alabama and clerking for Justice David Souter of the U.S. Supreme Court. His concurrence in the case of Snell v. United Specialty Insurance Company, — F.4th — 2024 WL 2717700 (May 28, 2024), unusual in its approach, aimed to pull back the curtain on how legal professionals can leverage modern technology to enhance judicial processes. It takes an optimistic and hopeful tone on how LLMs could improve judges’ decision making, particularly when examining the meaning of words.
Background of the Case
The underlying case involved a plaintiff (Snell) who installed an in-ground trampoline for a customer. Snell got sued over the work and the defendant insurance company refused to pick up the tab. One of the key questions in the litigation was whether this work fell under the term “landscaping” as used in the insurance policy. The parties and the courts had anguished over the ordinary meaning of the word “landscaping,” relying heavily on traditional methods such consulting a dictionary. Ultimately, the court resolved the issue based on a unique aspect of Alabama law and Snell’s insurance application, which explicitly disclaimed any recreational or playground equipment work. But the definitional debate highlighted the complexities in interpreting legal texts and inspired Judge Newsom’s proposal to consider AI’s role in this process.
Judge Newsom’s Proposal
Judge Newsom’s proposal is both provocative and forward-thinking, discussing how to effectively incorporate AI-powered LLMs such as ChatGPT, Gemini, and Claude into the interpretive analysis of legal texts. He acknowledged that this suggestion may initially seem “heretical” to some but believed it was worth exploring. The basic rationale is that LLMs, trained on vast amounts of data reflecting everyday language use, could provide valuable insights into the ordinary meanings of words and phrases.
The concurrence reads very differently – in its solicitous treatment of AI – than many other early cases dealing with litigants’ use of AI, such as J.G. v. New York City Dept. of Education, 2024 WL 728626 (February 22, 2024). In that case, the found ChatGPT to be “utterly and unusually unpersuasive.” The present case has an entirely different attitude toward AI.
Strengths of Using LLMs for Ordinary Language Determinations
Judge Newsom systematically examined the various benefits of judges’ use of LLMs. He commented on the following issues and aspects:
- Reflecting Ordinary Language: LLMs are trained on extensive datasets from the internet, encompassing a broad spectrum of language use, from academic papers to casual conversations. This training allows LLMs to offer predictions about how ordinary people use language in everyday life, potentially providing a more accurate reflection of common speech than traditional dictionaries.
- Contextual Understanding: Modern LLMs can discern context and differentiate between various meanings of the same word based on usage patterns. This capability could be particularly useful in legal interpretation, where context is crucial.
- Accessibility and Transparency: LLMs are increasingly accessible to judges, lawyers, and the general public, offering an inexpensive and transparent research tool. Unlike the opaque processes behind some dictionary definitions, LLMs can provide clear insights into their training data and predictive mechanisms.
- Empirical Advantages: Compared to traditional empirical methods such as surveys and “corpus linguistics”, LLMs are more practical and less susceptible to manipulation. They offer a scalable solution that can quickly adapt to new data.
Challenges and Considerations
But the Judge’s take was not all completely rosy. He acknowledged certain potential downsides or vulnerabilities in the use of AI for making legal determinations. But even in this critique, his approach remained optimistic:
- Hallucinations: LLMs can generate incorrect or fictional information. However, Judge Newsom argued that this issue is not unique to AI and that human lawyers also make mistakes or manipulate facts.
- Representation: LLMs may not fully capture offline speech, potentially underrepresenting certain populations. This concern needs addressing, but Judge Newsom stated it does not fundamentally undermine the utility of LLMs.
- Manipulation Risks: There is a risk of strategic manipulation of LLM outputs. However, this risk exists with traditional methods as well, and transparency in querying multiple models can mitigate it.
- Dystopian Fears: Judge Newsom emphasized that LLMs should not replace human judgment but serve as one of many tools in the interpretive toolkit.
Future Directions
Judge Newsom concluded by suggesting further exploration into the proper querying of LLMs, refining the outputs, and ensuring that LLMs can handle temporal considerations for interpreting historical texts (i.e., a word must be given the meaning it had when it was written). These steps could maximize the utility of AI in legal interpretation, ensuring it complements rather than replaces traditional methods.
Snell v. United Specialty Insurance Company, — F.4th — 2024 WL 2717700 (May 28, 2024)