Frontier Models Plateaued; Your Supply Chain Is Already Burning

Princeton's ICML 2026 paper added GPT 5.5, Gemini 3.5 Flash, and Claude Opus 4.7 to their reliability framework and found the same thing they found the last three times: zero meaningful improvement on multi-step agentic tasks across model generations. Hard-tier pass rates on the new ALE benchmark sit at 2.6%. SWE-Marathon, which tests coherence at billion-token budgets against real engineering work — Slack clones, JAX→PyTorch rewrites, C compiler builds — shows degradation well before the context ceiling. The finding is now peer-reviewed across all three major labs. This is not a Twitter narrative. The plateau is empirical.

Yes, but — a genuine reliability step from one lab in the next two quarters would kill this thesis on contact, and that has been the bear's graveyard before. What makes this audit different is that it holds across competitors optimizing against different objectives. Weight that.

The practical consequence: any roadmap item gated on "the next model will fix it" has no known delivery date. GitHub's CPO confirmed 17 million agent-generated pull requests in March 2026 alone — 3x their capacity forecast, driven by a December 2025 capability inflection that naive time-series modeling didn't survive. That volume is scaling on infrastructure built around the assumption that reliability improves. Princeton says it won't. The gap is widening every month agents run in production.

The Harness Is the Product Now

The same Princeton paper buried a result that deserves more attention: agents using hand-rolled raw API calls consumed 6x more tokens with lower success rates than agents using properly abstracted tooling. Tool design is a measurable performance lever. Verbose JSON schemas and chatty interfaces show up directly in token cost and in reliability numbers. This is the actual engineering investment that moves the needle — not waiting for a checkpoint.

Copilot's response to the 17M PR deluge validates the pattern from a different angle. GitHub's semantic routing between Flash-class and frontier models under an "auto" setting, combined with Cloudflare's per-model spend caps and automatic fallback, is now production-validated cost architecture. The arithmetic is concrete: reroute 10% of a $10M inference bill and you keep ~$1M. Under Copilot's June 1 switch to usage-based billing, that routing decision is a direct line item, not a latency optimization.

MiniMax M3 shipping open weights with a 1M-token context window, Gemma 4 QAT running in roughly 1GB via Unsloth's dynamic GGUF, and Kimi K2.5 posting agentic scores competitive with Opus 4.7 all land in the same week. The proprietary long-context moat is collapsing. Hybrid local/cloud routing — small local model produces answer plus uncertainty estimate, only the high-uncertainty tail routes to a frontier API — is a Q3 architecture decision, not a 2027 one. One implementation detail worth flagging: naive QAT→Q4_0 conversion via llama.cpp erases the accuracy gains QAT was designed to preserve. The Unsloth dynamic GGUF path handles it correctly. Teams benchmarking Gemma 4 with the default conversion path are underestimating real quality.

The Supply Chain Is on Fire

The reliability plateau story would be enough for one week. It's not the only story.

The Miasma worm compromised 73 Microsoft GitHub repositories across four organizations and remains uncontained. This is not another manually poisoned package — it's a self-replicating organism. Every CI run that pulls a contaminated dependency becomes a new propagation node. The IronWorm campaign running in parallel poisoned 50+ npm packages with a Rust-based information stealer targeting CI tokens, SSH keys, and .npmrc credentials — exactly the key ring needed to push poisoned versions further downstream. Your npm audit cannot see a compiled native binary. Static analysis is blind to it.

In the same window, an AI agent from an unnamed security startup found 21 zero-days in FFmpeg in a single research pass. A batch that size from one run is statistically out of distribution for agent-driven discovery to date. FFmpeg sits underneath torchvision.io, decord, PyAV, OpenCV video capture, and Whisper audio preprocessing. Any data loader running torchvision.io.read_video(url) on scraped content in the same process as the trainer is a live exploit path to wandb keys, S3 credentials, and model checkpoints. Patches are staging but not deployed.

The HuggingFace Transformers RCE deserves its own line: 2.2 billion installs, and the exploit path runs through model config files, not the pickle weights everyone has been warning about for two years. trust_remote_code=True auto-loading custom code from config.json is the likely vector. The real target is the research path — a data scientist evaluating ten candidate models pulls configs from ten untrusted repos onto a workstation with cached cloud credentials. Pin Transformers to the patched version and set trust_remote_code=False by default in all CI and production paths. Patching without auditing closes roughly half the exposure; the other half is sitting in configs already cached on disk.

Prompt Injection Has No Clean Fix

OpenAI shipped Lockdown Mode, which disables Deep Research, Agent Mode, web image fetching, and file downloads — entirely, not hardened. The best-resourced lab in the space decided the model-layer defense failed and the answer is feature amputation. Meta's AI chatbot was social-engineered into changing Instagram account emails via conversational prompt. No zero-day. No credential stuffing. A conversation. The chatbot performed the account takeover on the attacker's behalf.

Microsoft catalogued 7 new AI agent failure modes this week, extending the taxonomy well past prompt injection into systematic, reproducible attack categories. The pattern across every incident is identical: an LLM was given write access to a privileged system with no deterministic authorization gate. Better prompts don't fix this. The architectural fix splits the trust boundary — one model reads untrusted input and emits structured proposals, a separate deterministic path authorizes side effects. Any LLM-initiated mutation of user identity, credentials, or money requires an out-of-band verification the model cannot forge.

Claude Code's 7-tier permission model, including the "bubble" pattern where a subagent cannot authorize itself and must escalate to a parent, is the reference design worth stealing. It's UNIX process privilege inheritance ported to agents, and it's genuinely good architecture in a week of bad news.

What to Do This Week

Three things, in priority order.

First, run the supply chain audit today. Diff your SBOM against published Miasma and IronWorm IOC lists. For any npm package installed or updated in the last 14 days from the affected set: rotate npm publish tokens, GitHub PATs, CI runner cloud credentials, and any secrets those builds touched. Move all FFmpeg and video decode to a separate container with no IAM role, passing tensors over shared memory. Block direct HuggingFace Hub pulls from production and mirror approved models to a private registry with checksum manifests.

Second, add consistency@k (N≥5 trajectories per task) to your agent eval harness this sprint. Pass@1 masks the reliability plateau — variance across runs is the actual signal. Audit eval scaffolds for answer leakage while you're in there; if GAIA, maintained by researchers who study this exact problem, had leakage bugs, the harness written on a Tuesday afternoon does too.

Third, instrument cost-per-merged-PR and tokens-per-resolved-task before the Copilot usage-based billing creates a surprise invoice. The telemetry that makes a routing decision defensible in budget review is agent traces stored as first-class data, not stdout. Build the router after you can see what you're routing.