Frontier models stopped getting more reliable. Everything else got worse.

Princeton's updated ICML 2026 reliability audit dropped this week and it lands where the last three rounds landed: GPT 5.5, Gemini 3.1 Pro, Gemini 3.5 Flash, and Claude Opus 4.7 are not meaningfully more reliable than the models they replaced on agentic tasks. Three labs, three training regimes, one ceiling. The ALE benchmark that maps to the U.S. occupational taxonomy pegs the hardest tier at a 2.6% full-pass rate. SWE-Marathon shows coherence collapsing well before the 1B-token budget is spent.

In the same news cycle, GitHub's CPO confirmed 17 million agent-authored pull requests in March 2026 alone — roughly 3x the capacity plan, enough to saturate their West Coast network into an emergency Azure migration. Copilot flipped to usage-based billing on June 1. Anthropic says Claude writes over 90% of its production code.

Hold those two numbers in one head. Reliability is flat. Volume is compounding. If the roadmap has features sequenced behind "ship when the next model gets better," that sequencing is dead this sprint.

Yes, but — a genuine step from GPT-6 or an equivalent Opus release would reassert the frontier premium overnight, and Princeton is measuring today's models, not next quarter's. Fair. The step hasn't arrived in four quarters, and Anthropic simultaneously filing an S-1 and calling for a global pause isn't the posture of a lab about to ship a discontinuity.

The harness is where the gains are hiding

Hugging Face's CEO put a number on the tooling delta: hand-rolled raw-API agents burn 6x more tokens with lower success rates than purpose-built CLI wrappers. Good tools are cached intelligence. Encode the domain logic, the validation, and the workflow shape into the interface, and the model doesn't have to reason its way there. Claude Code's 7-tier permission architecture, with an ML classifier deciding when to prompt for approval, is the reference design — a meta-AI layer that every agent product eventually needs.

So if you're a builder reading this on Sunday: two diagnostics before Monday's standup. When a workflow fails, is it a reasoning failure or an orchestration failure? And is unit cost dominated by tokens, retries, or human review? Tokens point to caching. Retries point to validators. Review points to narrower scope. None of those cells wait for a model release.

Add consistency@k across N≥5 trajectories to your eval harness this sprint. Single-shot pass@1 was never the metric that mattered in production, and Princeton just made that legible. While you're in there, grep your scaffolds for ground-truth leakage into agent context — Princeton found it on GAIA, which means your internal harness almost certainly has the same bug.

The stack you build on is the attack surface

The reliability ceiling would be its own story. It isn't the only one.

CVE-2026-20245 in Cisco Catalyst SD-WAN Manager is under active exploitation with no patch. The Miasma worm self-replicated across 73 Microsoft GitHub repositories in four organizations, and IronWorm poisoned 50+ npm packages with a Rust info-stealer harvesting CI tokens, .npmrc credentials, and SSH keys — the exact set needed to push further downstream. Hugging Face Transformers, at 2.2 billion installs, has an RCE reachable through model config files, not weights. The trust boundary most teams drew — prefer safetensors, avoid pickle — is now necessary but insufficient, and the target is the research workstation where a data scientist evaluates ten candidates in an afternoon on a machine with cached registry and cloud credentials.

An AI agent autonomously found 21 zero-days in FFmpeg in one research cycle. FFmpeg sits under torchvision.io, decord, PyAV, OpenCV, and Whisper preprocessing. If your data loader runs torchvision.io.read_video(url) on scraped content in the same process as the trainer, a malicious MP4 reaches your W&B keys.

Meta's AI chatbot was talked into changing the email on high-profile Instagram accounts. No credential stuffing, no technical bypass — a conversation. The model did exactly what it was authorized to do. That's a confused deputy by construction, and it's the pattern OpenAI just conceded with Lockdown Mode: Deep Research, Agent Mode, web image fetching, and file downloads aren't hardened, they're disabled. When the lab with the most prompt-injection telemetry on the planet ships their fix as an off-switch, stop pretending your guardrails are doing the job. Gate capabilities. Don't classify intent.

Map every LLM tool along two axes: does it ingest untrusted content, does it perform privileged actions. Anything in the intersection needs capability ablation or out-of-band verification. If the policy can be talked out of its decision in English, it's not a policy — it's another model.

What to do this week

Three things, in order.

First, patch and rotate. Pin Transformers, set trust_remote_code=False by default in CI, restrict Cisco SD-WAN Manager admin access to jump-host-only, and rotate npm publish tokens, GitHub PATs, and cloud CLI credentials for any developer who installed npm packages in the last 30 days. Sandbox FFmpeg into a subprocess or container with no IAM role. This is 48 hours of work and it closes the live bleeding.

Second, instrument the harness. Add consistency@k and one long-horizon coherence task to your agent eval by end of sprint. Add prompt-injection and confused-deputy regression tests modeled on the Meta email-change attack. Cohort cost-per-merged-PR and tokens-per-resolved-task by agent vs. human author — pooled metrics are already masking divergent error distributions.

Third, kill one API dependency. Pick one workload — reranking, classification, tool-routing — and spike Gemma 4 QAT via Unsloth's dynamic GGUF path. It runs in roughly 1GB. If it works, you've bought vendor leverage and a hedge against the next Anthropic pricing letter. If it doesn't, you've measured your actual quality floor, which is more than most teams can say.

The model isn't coming to save the harness. The harness is the product.