The week LLM refusal stopped counting as a security boundary

Three admissions landed inside seven days, and they say the same thing.

Meta's Instagram AI chatbot was talked into changing the registered email on high-profile accounts. No exploit chain. No credential stuffing. An attacker asked the model to do something the model was permitted to do, and it did. Hugging Face Transformers — 2.2 billion installs — has an RCE that fires from config.json, not pickle weights, on any from_pretrained() call with trust_remote_code=True against an untrusted repo. And OpenAI shipped Lockdown Mode, whose mitigation for prompt injection is to turn off Deep Research, Agent Mode, image fetching, and file downloads. The lab with the deepest red team in the industry chose amputation over defense.

Read together, this is the vendors conceding in production what a lot of us have said in review: the model's refusal behavior is not an authorization boundary. It never was. The Instagram case is the cleanest proof — that wasn't even prompt injection. The model held a service scope broad enough to mutate identity state, and English was the API.

Yes, but — a fair counter is that Lockdown Mode is a temporary posture and the research pipeline behind constitutional classifiers, activation steering, and the sparse-autoencoder work will close the gap within two model generations. Maybe. The steady-state opt-in rate after the next release is the metric worth watching. Until it holds above 50% under product pressure, the take stands: capability scoping is the only control that has ever worked, and the vendors just said so with their roadmaps.

The pattern, and the fix that already shipped

The attack surface is now the artifacts and toolchains you trust by default. Model configs. MCP servers. Agent tool bindings. Claude Code's MCP has an actively exploited flaw in the connector layer — the thing that hands a model your ~/.aws/credentials and your shell. HuggingFace's config-RCE hits the worst-instrumented boxes in most estates: GPU inference nodes and research workstations with cached cloud creds. Microsoft added seven new agent failure modes to its taxonomy the same week, which means every agent shipped in the last twelve months was threat-modeled against an incomplete framework.

The design answer already exists. Claude Code's seven-tier permission model — enterprise policy over CLI flags over project over user over session grants over default-deny, with a bubble mode that escalates to parents instead of granting session-wide trust — is the reference implementation for graduated autonomy. Steal it. The one caveat: the auto tier uses an ML classifier to decide when to prompt, which means the classifier's false-negative rate is now part of your threat model. Log it, monitor it, don't pretend it's deterministic.

Two modes to ban outright on any host with production credentials: bypassPermissions and dontAsk. These delegate shell execution to a model whose input channel includes every README, issue comment, and dependency description in the repo. Enforce it via MDM fingerprinting, not via a wiki page.

The volume problem the security model wasn't built for

GitHub processed 17 million agent-authored pull requests in March. Their capacity plan called for 5% growth and got 15% — a 3x miss traced to the December 2025 macro-delegation inflection. The West Coast network saturated. Emergency Azure migration. Copilot moved to usage-based billing on June 1.

This is where the security story and the cost story fuse. A stolen PAT under seat pricing leaked code. Under usage-based billing, a stolen PAT looped through an agent endpoint with semantic routing free to escalate to Opus-tier models is financial denial-of-service with an invoice attached. Most SOCs don't alert on Copilot spend anomalies today. They will, after the first six-figure surprise.

The review-gate math also broke. Branch protection built around a human reviewer per PR is now a budgeting problem, not a policy. The controls that scale are the boring ones: SAST plus secret-scan plus SCA plus dependency-confusion checks, mandatory on any PR from an agent identity, no exceptions, no override without a ticket. Chronicle — GitHub's session-analytics product — persists prompts and generated code to GitHub-managed cloud. If your regulated business units enabled it without a DPIA, that's a SOC 2 finding waiting to be written.

One pattern to internalize: agents retrying against a slow queue produce exponential load, not linear. Same shape as a thundering-herd reconnect, one layer up. Rate-limit per agent identity and surface queue depth as backpressure. Agents that can't see the queue retry blind, and the retry storm shows up as your compute bill before it shows up as an incident.

What to do this week

Pick one thing and finish it by Friday. Enumerate every LLM-fronted flow in your environment that can mutate identity state — email, phone, MFA, password recovery, account merge, billing address. Any surface meeting three conditions is reproducibly vulnerable to the Meta attack: the LLM has access to identity-mutating APIs, it accepts input from an untrusted party, and there is no out-of-band verification between the model's decision and the action. Put a WAF or API-gateway rule in front of each one that forces a cryptographic challenge, a re-auth, or a human approval before the mutation lands. That single control breaks the chain.

Everything else — the HF pin, the MCP allowlist, the branch-protection rule for agent PRs, the Chronicle DPIA, the ban on bypassPermissions on prod-credentialed hosts — sits on the same architectural premise: the model proposes, a deterministic layer the model cannot argue with disposes. Anything less and the first postmortem you write this quarter will be adversarial.