The 80% Problem: Your AI Stack Is One Empty Filter From Production

Two numbers from this week deserve to sit on the same whiteboard.

The first: a preregistered Wharton study with 1,372 participants and roughly 10,000 trials found that when AI gives a wrong answer, users follow it 80% of the time. Not 30%. Not 50%. Eighty. Of those follows, 73% were pure cognitive surrender — no override attempted, no friction, no second look. Effect size was Cohen's h of 0.81, which in social science terms is a building falling on you. The strongest predictor of surrender wasn't task difficulty or model accuracy. It was trust in AI, with a 3.5x odds multiplier. Your power users — the ones generating your best engagement charts — are statistically the most likely to ship the bad output.

The second: on February 20, a Cloudflare cleanup task ran with an empty pending_delete parameter. The system read "empty" as "match all," queued 4,306 BYOIP prefixes for deletion, and withdrew about 1,100 BGP routes — 25% of all BYOIP routes on the network. Six-hour outage. 1.1.1.1 returning 403s. Magic Transit dark. No attacker. No zero-day. Just an unbounded operation hitting a destructive code path with no upper limit on how much it was allowed to break.

These are the same failure. One is a human accepting an unbounded recommendation from a model. The other is a script accepting an unbounded command from a query. In both cases, the missing primitive is a blast-radius cap.

The pattern repeated three more times this week

AWS confirmed multiple late-2025 outages caused by internal AI tooling — outages employees described as "entirely foreseeable." Amazon's Kiro agent autonomously deleted and recreated an environment, taking down a service for 13 hours. Hudson Rock confirmed the first commodity-malware extraction of a complete agent identity from an OpenClaw instance: token, keys, the soul.md instructions, the memory log. 135,000+ OpenClaw instances are exposed on the public internet and 63% are flagged vulnerable. The Cline supply-chain attack rode a prompt injection into an npm publish token and shipped malicious packages for eight hours.

Not one of these required a sophisticated adversary. They required someone to ship a system that could do an arbitrary amount of damage when something — a query, a model, a malware sample, a prompt — got it wrong.

The Wharton finding is an architecture problem, not a training problem

The instinct from leadership when you show them the 80% number will be to suggest training. More AI literacy. Better onboarding. A Slack channel about prompt hygiene. None of that works, and the study tells you why: confidence in AI output goes up when accuracy goes down. Users borrow the model's certainty without inheriting its error rate. The fix has to be structural.

The one cohort in the study that performed identically to the no-AI control was the "Independents" — people who reasoned first, then consulted the model. That's the only intervention that broke the surrender pattern. Which means the design move is straightforward: make the user commit to an answer before the model speaks. Hide the suggestion behind a hypothesis. Require a written justification when AI output is accepted on a high-stakes path. In code review, switch AI tools to flag-only mode and remove the auto-suggested fix.

This is the same logic as Cloudflare's post-mortem remediation: circuit breakers above a threshold, mandatory dry-run on destructive batches, health-mediated config snapshots, separation of operational and desired state. You don't trust the operation just because it's authorized. You trust it because it can't blow past a defined limit.

What to instrument this week

Grep your infrastructure code for destructive verbs — delete, destroy, withdraw, revoke, purge — and for each, answer one question: what's the maximum number of resources this can touch in a single run? If the answer is "all of them," or "however many match the filter," that's the Cloudflare bug, sitting in your codebase, waiting for an empty parameter. Add a hard cap. Five percent of the resource pool is a defensible default. Above that, require a dry-run output and a human approval gate. Empty filters on destructive paths should reject, not match-all.

Then do the same audit on your AI-assisted surfaces. For every place a model output flows into a user decision or a system action, ask: where's the cap? Where's the think-first gate? Where's the override-rate metric that tells me whether my analysts are actually reviewing or just clicking accept? If your AI feature has an auto-approve rate above 70% and you don't know whether the model is right that often, you're shipping the Wharton study into production.

One metric to start tracking on Monday: AI override rate, broken out by user trust segment. If your highest-trust users are overriding less than 15% of the time, you don't have a power-user cohort. You have the cognitive-surrender cohort, and the 3.5x odds multiplier says they're the ones who'll wave through the bad output that takes you down.

The 80% number and the empty-filter outage are both telling you the same thing. The era of unbounded operations — by humans, by scripts, by agents — is over. Cap the blast radius before something cheaper than an attacker finds it for you.