Signal vs Noise 001 | AI Made a Lawyer Lie to a Judge. Now What?
When the system that's supposed to find truth starts hallucinating, who checks the checker?
A prosecutor in Wisconsin submitted court documents in February 2026.
The citations looked real. The case references looked legitimate. The arguments were structured and confident.
The judge checked.
None of the cited cases existed.
The prosecutor had used AI to draft the documents. The AI had hallucinated — invented cases, fabricated references, constructed a legal argument built entirely on fiction. The documents were thrown out. The prosecutor was sanctioned.
This wasn't a one-off.
Over 160 documented cases of AI hallucinations in courtrooms have been recorded since 2023. Lawyers fined. Documents rejected. In California, courts have gone further — not only sanctioning the lawyer who submitted false citations, but also the opposing lawyer who failed to catch them.
The system is now holding everyone responsible for what the AI gets wrong.
This is not a technology problem.
When people talk about AI hallucinations, they frame it as a bug — something to be fixed in the next model update, a technical limitation that will eventually be resolved.
But what happened in those courtrooms is not a technology problem.
It's a trust problem.
The lawyers didn't lie intentionally. They trusted the output. They assumed that because the AI produced something that looked structured, cited, and confident — it was real.
That assumption is the problem.
We have built systems that are extraordinarily good at producing the appearance of truth. Fluent sentences. Logical structure. Correct formatting. The right tone for the right context.
What they cannot guarantee is the thing underneath all of that: whether any of it actually happened.
The signal was always there.
Here's what's interesting about the RealityCheck legal tool that emerged in response to this crisis.
It doesn't produce truth. It checks for the absence of it.
It looks at AI-generated documents and flags where the citations don't exist, where the logic doesn't hold, where the confidence of the output has outrun the evidence behind it.
In other words: it calibrates.
This is the function that got removed when we handed the drafting over to AI. Not the writing. Not the formatting. Not the speed.
The calibration.
Someone still has to know what real looks like — in order to recognise when it's missing.
Why this matters beyond the courtroom.
A lawyer submitting fake citations to a judge is a visible, documentable failure. There's a record. There's a sanction. The error is caught — eventually.
But most AI hallucinations don't happen in courtrooms.
They happen in medical summaries. In financial reports. In news articles. In the analysis that gets forwarded, cited, built upon, until the original hallucination is so embedded in the chain that no one can find where it started.
The courtroom cases are not the problem.
They are the first visible symptoms of a much larger structural issue: we have deployed systems that produce confident-sounding outputs at scale, and we have not kept pace with the infrastructure needed to verify them.
The noise is winning. For now.
The calibrators are coming.
In 2023, the problem was identified.
In 2024, the tools started appearing.
In 2025, the courts started enforcing accountability.
In 2026, the legal profession is beginning to accept that AI output requires human verification — not as an optional extra, but as a professional obligation.
This pattern will repeat in every field where AI output has consequences.
Medicine. Finance. Education. Journalism. Policy.
The question is not whether calibration will be required. It's who will be qualified to provide it.
The people who understood early that noise and signal are not the same thing — who built frameworks for telling them apart, who accumulated real cases and real observations rather than generated content — those people are not ahead of the curve.
They are the curve.
A note on the name.
There is now a legal tool called RealityCheck.
It does one thing: it tells lawyers when their AI-generated documents contain content that isn't real.
We've been doing this longer, and for a wider problem.
Welcome to the conversation.
Next in Signal vs Noise: The Algorithm Says It Wants Real. But Can It Tell the Difference?
→ [Link to Part 2 when published]
Signal vs Noise | 001
AI 讓律師對法官說了謊。然後呢？
當本來應該找出真相的系統開始產生幻覺，誰來校準校準者？
2026年2月，威斯康辛州一位檢察官提交了法庭文件。
引用看起來真實。案例參考看起來合法。論點結構清晰，語氣自信。
法官去查了。
那些引用的案例，一個都不存在。
檢察官用了AI來起草文件。AI產生了幻覺——憑空捏造案例，虛構引用，建構了一個完全建立在虛假之上的法律論點。文件被駁回。檢察官被制裁。
這不是個別事件。
自2023年以來，法庭上已記錄超過160宗AI幻覺案例。律師被罰款。文件被拒絕。加州法院更進一步——不只制裁提交虛假引用的律師，連對方律師沒有發現並舉報，也被認為有責任。
系統現在要求所有人，都要為AI的錯誤負責。
這不是技術問題。
當人們談論AI幻覺，他們把它框架為一個漏洞——等待下一個版本修復的技術限制，一個最終會被解決的問題。
但在那些法庭上發生的事，不是技術問題。
是信任問題。
那些律師不是故意說謊。他們信任了輸出結果。他們假設，因為AI生成的東西看起來有結構、有引用、有自信——它就是真實的。
這個假設，才是問題所在。
我們建造了一個系統，它極其擅長製造真相的外觀。流暢的句子。邏輯的結構。正確的格式。符合場景的語氣。
它無法保證的，是這一切之下的那件事：這些內容是否真實發生過。
訊號一直都在。
關於回應這場危機而出現的RealityCheck法律工具，有一件有趣的事。
它不生產真相。它檢查真相的缺失。
它審視AI生成的文件，標記出不存在的引用、站不住腳的邏輯、輸出的自信已經超越其背後證據的地方。
換句話說：它在校準。
這正是我們把起草工作交給AI時，被移除的那個功能。不是寫作。不是格式。不是速度。
是校準。
仍然需要有人知道真實是什麼樣子——才能在它缺席的時候認出來。
為什麼這件事的影響遠超法庭。
律師向法官提交虛假引用，是一個可見的、可記錄的失敗。有記錄。有制裁。錯誤最終被發現。
但大多數AI幻覺，不發生在法庭上。
它們發生在醫療摘要裡。在財務報告裡。在新聞文章裡。在那些被轉發、被引用、被層層疊加的分析裡——直到原始的幻覺已經深埋在鏈條之中，沒有人能找到它從哪裡開始。
法庭案例不是問題本身。
它們是一個更大結構性問題的第一批可見症狀：我們大規模部署了能夠生成自信聽起來的輸出的系統，卻沒有跟上驗證它們所需的基礎設施。
噪音正在佔上風。暫時如此。
校準者正在到來。
2023年，問題被識別。
2024年，工具開始出現。
2025年，法院開始執行問責。
2026年，法律界開始接受一件事——AI輸出需要人類驗證，不是可選的附加項，而是專業義務。
這個模式將在每一個AI輸出有後果的領域重複——
醫療。金融。教育。新聞。政策。
問題不是校準是否會被要求。而是誰有資格提供它。
那些早早就理解了噪音和訊號不是同一件事的人——那些建立了分辨框架、積累了真實案例和真實觀察而非生成內容的人——
他們不是走在曲線前面。
他們就是那條曲線。
關於這個名字。
現在有一個法律工具叫做RealityCheck。
它只做一件事：告訴律師，他們的AI生成文件包含了不真實的內容。
我們做這件事更久，針對的是更廣泛的問題。
歡迎加入這場對話。
Signal vs Noise 下一篇：演算法說它想要真實。但它能分辨嗎？
→ [連結至第二篇]
