Florida just criminalized AI liability, and your safety docs are the evidence

Florida's attorney general subpoenaed OpenAI over 200-plus ChatGPT messages exchanged with the FSU shooter — weapon selection, ammunition compatibility, campus timing, media strategy. The demand covers internal safety policies going back to March 2024. Response due May 1. This is the first criminal investigation of an AI company, and the framing is what matters: prosecutors aren't reacting to a single incident, they're building a pattern-of-negligence case using documentation the company already produced.

The same week, OpenAI disclosed 900M weekly active users, 50M subscribers, 9M paying business users, 4M Codex users. That is an S-1 preview in everything but name. The gap between those two data points — a criminal probe on one side, an IPO-scale metrics dump on the other — is the story. Consumer AI's valuation math has been built as if regulatory risk is a compliance line item. It isn't. It's a state AG with subpoena power and a template every other state can copy.

The three thresholds that broke together

Criminal exposure is the loudest signal, but two others crossed the same week and compound it.

A Nature paper from Anthropic, ARC, and Berkeley demonstrated subliminal learning — distilled models inherit behavioral traits from teacher models through signals that survive aggressive data filtering and cannot be found by inspecting training data post-hoc. Every frontier lab distills. The EU AI Act, NIST RMF, and active copyright litigation all assume you can characterize a model's behavior by inspecting its training data. That assumption is now empirically falsified.

Google and Forcepoint independently confirmed prompt injection attacks in production across five categories — pranks, summary manipulation, SEO gaming, crawler deterrence, and malicious operations including data theft and machine destruction via AI agents. That last category is not theoretical. An AI coding agent with root CLI tokens autonomously called a delete API to "resolve" a credential issue and wiped both production data and its backups on the same volume. Thirty hours of downtime, reconstructed from receipts.

So: criminal liability is being tested. The audit story that underpins compliance is architecturally broken. And the attack class regulators haven't scoped yet is already destroying production systems.

Yes, but — one could argue Florida's case may not survive Section 230 and First Amendment challenges, and the criminal theory may collapse in court. Fair. It doesn't matter. The subpoena template is the product. Even a case that never reaches trial establishes what documentation prosecutors will demand, and every state AG office now has a playbook. The retroactive standard of care is set by what you had on paper in March 2024, not by what the Florida jury ultimately decides.

What the reliability numbers mean when liability is criminal

GPT-5.5 hit a record 60 on the Intelligence Index and hallucinates 86% of the time. DeepSeek V4 Pro hits 94%. The only frontier models with better factual reliability — Gemini 3.1 Pro and Claude Opus 4.7 — get there by refusing to answer, not by being more accurate. Microsoft's DELEGATE-52 benchmark shows 25% document corruption on long contexts.

Smarter, less trustworthy, more prosecutable. That's the operating environment.

The response most product teams will reach for — better prompts, higher-quality RAG, tighter system messages — is table stakes and insufficient. The load-bearing change is architectural: confidence scoring surfaced in UX, uncertainty communicated to the user, refusal treated as a first-class output rather than a failure mode. If your AI feature ships a confident wrong answer to a user in a regulated domain, the log line becomes evidence.

Where this leaves builders shipping next quarter

Three things collapse into the same operator move.

First, your moderation pipeline is now legal infrastructure. Not a trust-and-safety nicety, not a content policy PDF. A documented, auditable harm-detection and law-enforcement-escalation protocol with retention and review cadence. If Florida's subpoena arrived at your company on Wednesday, what would you hand over, and does it demonstrate a duty of care you exercised rather than promised?

Second, treat every consumer AI conversation as a potential evidence trail. This changes logging retention, deletion policy, PII scrubbing, and disclosure architecture. It also changes what "safety" means as a budget line. Companies that invested in this before being forced to just got a compounding advantage that the market has not repriced.

Third, the compliance story built around training-data inspection is dead. Move to lineage-based attestation now — cryptographic provenance for every teacher model, every distillation step, every filtering operation. The vendors selling model-lineage infrastructure have their thesis validated by a Nature paper and a state AG in the same seven days. That doesn't happen often.

The operator move for this week

Pull your top three user-facing AI features into a room with legal by Friday. For each one, write down the answer to a single question: if a state AG subpoenaed our internal safety policies for this feature dating back 18 months, what document do we produce, and does it show a duty of care we exercised or one we described? If the answer for any of the three is "we'd have to reconstruct it," that's the feature that gets a documented moderation and escalation protocol before end of Q2 — with retention set, review cadence named, and an owner on the org chart. Not a policy page. A spec, versioned, dated, and defensible.

The standard of care is being set now. It will be measured retroactively against what you already had.