11. What Happens When You Cite AI-Fabricated Case Law As-Is
Contents
More lawyers are getting sanctioned every year for filing AI-fabricated case law with courts. What courts punish isn't the fact that 'AI was used' — it's the fact that the result was submitted without being checked.
Background — Why This Topic, Why Now
In June 2023, New York federal judge P. Kevin Castel fined lawyers Steven Schwartz and Peter LoDuca $5,000 each. They had submitted several ChatGPT-fabricated airline lawsuit cases to the court as-is. (ABA Journal) This case, Mata v. Avianca, is often called the starting point of the 'AI hallucinated case law' problem — but it was far from the end.
A database of AI hallucination cases run by HEC Paris researcher Damien Charlotin collects only cases where courts confirmed that AI-fabricated content was actually cited. This tally grew from about 200 cases in mid-2025 to 1,600 cases across 35 countries by June 2026 — nearly an eightfold increase in a single year. (Malay Mail) This isn't happening because lawyers don't know better. AI hallucination is already a widely known risk, yet the same mistake keeps repeating.
The same pattern shows up outside law, too — from fake citations in academic papers to fake book recommendations in news articles. The structure is identical: AI plausibly fabricated something, and a human sent it out without checking.
Key Data & Current State
| Field | Case | Outcome / Action | Date |
|---|---|---|---|
| Law (US) | Mata v. Avianca | Two lawyers fined $5,000 each | 2023.06 |
| Law (US) | Johnson v. Dunn (N.D. Ala.) | Three law firm lawyers publicly reprimanded, barred from the case, referred to state bar for discipline | 2025.07 |
| Law (worldwide) | AI hallucination case database (Charlotin, crowdsourced) | 1,600 cumulative cases across 35 countries | 2026.06 |
| Academic publishing | Lancet analysis of PubMed-indexed papers | 1 in 277 papers contained fabricated citations (sharp rise from 1 in 2,828 in 2023) | 2026.05 |
| Media | Chicago Sun-Times summer reading list feature | 10 of 15 recommended books confirmed not to exist; feature pulled and apologized for | 2025.05 |
Interpretation: Law, academia, and media all converge on the same failure — "the AI's output was plausible, so the check got skipped." The penalties and responses differ by field, but the cause is one and the same.
In-Depth Analysis
1) The Line for Punishment Is 'Didn't Verify,' Not 'Used AI'
In Johnson v. Dunn, Judge Anna Manasco sanctioned three lawyers who had filed a brief containing five ChatGPT-fabricated citations — a public reprimand, removal from the case, and even a referral to the state bar for discipline. This happened even though the firm already had its own AI usage policy and warning procedures in place. (EDRM) The court didn't take issue with the lawyers using AI tools themselves. What it cited was their failure to fulfill the duty to verify the truth of what was in the brief — the court characterized this as "recklessness in failing to verify the truthfulness of representations."
This standard connects to the signature-responsibility principle under Rule 11 of the US Federal Rules of Civil Procedure. By signing, a lawyer certifies that they have reasonably investigated the content of every brief filed with the court. Whether AI drafted it or a person wrote it by hand, the responsibility borne by the signer is the same.
2) Why It Keeps Happening — The Trap of 'Plausibility'
AI-fabricated case law comes dressed in the exact same format as real cases — case name, year, court, citation style, all present. Only the content is fake; the format is flawless, so a routine skim doesn't catch it. Charlotin's database is heavily mixed with cases from pro se litigants representing themselves, but it also steadily includes cases filed by lawyers. That means even experts with legal knowledge keep making the same mistake.
The more fundamental problem is that the actual cost of verification isn't high at all. Whether a single case actually exists can be confirmed within minutes just by searching the case name and citation number in an official case law database. This is exactly why courts keep demanding it not happen again — not because verification is hard, but because the verification step itself was skipped entirely.
3) Not Just a Legal Problem — The Same Failure in Journals and Media
The same structural problem is growing in medical and scientific journals too. An analysis published in The Lancet reviewed roughly 2.5 million PubMed-indexed papers and reported that the share of papers containing fabricated citations had risen more than tenfold in 2026 compared to 2023. (Retraction Watch) One journal cross-checked all 117 footnotes in a published paper, confirmed 6 were AI-fabricated citations, and even discovered that all of the fake citations had been added after the initial peer review — leading it to retract the paper.
Media had a similar incident. A summer reading list feature published by the Chicago Sun-Times in 2025 included 10 nonexistent books out of 15 recommendations. The article had been outsourced by an external content provider to a freelance writer, who admitted to passing along an AI-generated list without verifying it. (404 Media) The structure of the incident is the same across all three fields: the person who ultimately put their name on AI-generated output never checked it.
4) Why 'Failure to Verify' Gets Punished
The reason professions like lawyer, researcher, and journalist exist at all is that the person vouches for the accuracy of their own output. AI cannot stand in for that guarantee. What AI produces is still just a draft, and the moment someone puts their name on it, responsibility shifts entirely to that person.
Analysis: Courts and journals asking "did you verify it" rather than "did you use AI" amounts, in the end, to a procedure for reconfirming who holds that guarantee of responsibility. How much of the work AI took on doesn't change where the responsibility lies.
In Practice — Don't Cite It Until You've Verified the Source Yourself
- Treat any citation, figure, or case AI gives you as 'nonexistent' until you've opened the original source and confirmed it yourself. No exceptions, even when the format looks correct.
- Even if the citation is real, you're not done. Check the original source directly to confirm it actually supports what you're claiming. There are sanctioned cases where a real case was simply cited in the wrong context.
- Don't ask the AI itself, "is this really real?" The same model can fabricate another piece of made-up evidence and answer "yes." Verification must always happen against the primary source.
- Don't use anything you don't have time to verify. Don't hedge with phrases like "it is reportedly..." — if you couldn't confirm it, just leave it out.
- Establish the principle that whoever puts their name on the output is the final verifier. Even if AI wrote the draft, the person who reviewed and signed off bears responsibility.
References
Contents
Related posts

API — What Does 'Plugging Someone Else's AI into Your Service' Actually Mean?
Connecting AI 'via API' does not mean bringing the model itself in-house; it means contacting someone else's server on every request to fetch an answer. This article lays out what to judge in practice, from token-based pricing and rate limits to API key security.

Vector DB — A purpose-built search warehouse for AI
A vector DB converts text and images into numeric coordinates (embeddings) to quickly find semantically similar items. It’s core to RAG, but only makes sense after weighing scale, latency, and search goals.

Training vs. Inference — When Does AI Actually Get Smarter?
It feels like AI gets smarter the more you talk to it, but it does not actually learn anything new mid-conversation. This article lays out how training and inference differ, and why test-time compute became the AI industry's new buzzword in 2026, with data.

Transformer — The Ancestor of Every AI Today
From GPT to Gemini to Claude, every major AI model in 2026 runs on the Transformer architecture Google published in 2017. This article covers how self-attention works and the line of variations from decoder-only models to MoE and Mamba hybrids.