Microsoft Down 21% YTD, AI Scorecard Is Copilot Paid Seats

Why the market picked a seat count

An analyst opened Microsoft's earnings materials this week and reached for one number out of dozens: Microsoft 365 Copilot subscriber counts. Not because seats measure AI value well. Because a paid seat is the only AI figure an outsider can verify: a buyer chose to pay, per user, again. Usage minutes, prompt volume and "AI-assisted workflows" are constructions the vendor controls, so the market discounts them to zero. The Information's figures are pre-report; the earnings prints are the confirmation.

That question travels downward faster than most product teams expect. It lands in a roadmap review as "what did the AI feature convert?" — and an engagement chart is not an answer to it.

The same audit, pointed at product claims

Two examples from very different places share one structure. The feature was marketed on the easy metric while the hard one stayed unowned. The Bear Cave surfaced Forbes reporting that Axon's AI-generated police reports get facts wrong in public records, while the product is sold on time saved. Time saved is real. It is also not the gate. Once generative output enters a system of record, the binding number is the factual error rate, and nobody had published one. Tesla's Robotaxi line is the metric version of the same failure: "scaling," against operating miles that fell from 1.05M in Q1 to 0.75M in Q2, roughly a 30% decline.

An efficiency claim that has never been paired with an error rate is not a product metric. It is an unpriced promise.

Where the sources disagree

The Information frames AI capital as under-returning. TheSequence shows the opposite one layer down: Google Cloud growing 82% to $24.8B on enterprise AI demand, Alphabet planning roughly $180–190B of capex, and private marks re-rating hard, with Databricks moving from $134B to $188B in five months. Both readings hold. The reconciliation is the useful part. Infrastructure demand is genuine; per-feature paid conversion is what remains unproven. Apple is the uncomfortable control group, up 23% year to date on a real iPhone upgrade cycle with minimal AI spend. Having no AI story was not punished.

The move

Build the internal version of the seat metric before someone hands you theirs. Four numbers per AI feature: paid attach rate, retention delta between feature users and non-users, cost per successful outcome, and a documented error rate with its review policy. Most teams can assemble the first three from existing telemetry. Almost none have the fourth, which is exactly why it is the one that decides the launch review.

The mechanism behind the bug tax

An agent produces a diff, the tests pass, it merges. That is the moment it is optimizing for. Coding agents optimize for short-term correctness: whether this diff passes, whether this test goes green. They do not optimize for whether the code stays cheap to change in six months. Remove the review gate and that bias compounds directly into the codebase. The field pattern reported by HumanLayer's Dex is specific: "more comments, longer comments, tons of PRs merged with no review," with incidents and bugs per developer climbing sharply.

Watch the denominator. It is per developer, not per pull request. Throughput went up, which is what the autonomy demo shows. Defect load per human went up too, which is what the on-call rotation feels. Both numbers are true at once. Only one of them is in the vendor deck.

The benchmarks are conceding the point

Two evaluations, SWE-Marathon and Frontier Code, measure long-horizon work rather than single-shot patch correctness. That is the industry admitting its previous metric missed what production software actually needs: maintainability across many sessions. Separate the thing being pitched from the thing being done. Fable is shipping a "coach" collaboration model, where the agent proposes and the human decides, as a product pattern rather than a safety disclaimer.

Human review is not a limitation you apologize for in the demo. It is the only feature in this category that a buyer can currently verify.

Why buyers will ask about this in the next security review

Two items turned agent governance from a slide into a questionnaire. Zenity disclosed AgentForger, a cross-site request forgery flaw in OpenAI's Workspace Agents that lets an attacker forge a persistent malicious agent inside a tenant, which then exfiltrates that tenant's data. The attack creates an agent identity, not just a session. Separately, Risky.Biz documents that OpenAI's own containment lapse ran for roughly six days before anyone noticed, and the vendor learned of it from the victim's public write-up rather than its own monitoring.

Read those together and two questions land on any product in enterprise diligence: how is an agent's identity authorized, and how quickly would anyone know if one went rogue. TheSequence's framing is the right one for a PRD: least-privilege scoping, isolation and human-in-the-loop gates belong in the spec before development, not in the postmortem.

What to build

Three concrete additions, all cheap now and expensive later. First, a named reviewer in the flow for any agent action that writes to a shared system — coach mode, not a confirmation dialog nobody reads. Second, acceptance criteria covering agent identity, permission scoping, egress monitoring and audit-log retention, so the security answer is a document rather than a meeting. Third, track bugs and incidents per developer for agent-assisted work as a first-class quality metric, because that is the number that tells you whether the autonomy is compounding value or debt.

Start with your own routing bill

Picture the feature you shipped last quarter that sends arithmetic to a frontier thinking model because that was the safe default the week before launch. That is the real question here. Not research. Routing: which production features send structurally simple work — arithmetic, structured extraction, templated reasoning — to an expensive model out of habit. Rank the top three by monthly spend. That list is the only place the numbers below can pay.

The mechanism decides where the result applies, so read it precisely. A Qwen3-4B running on a $429 GPU answers basic arithmetic correctly 32% of the time. Probe its internal states and the correct answer is present about 80% of the time. So the failure is not capability. The model locks into a formatting pattern and presents instead of answering — autoregressive lock-in, meaning it commits early to the shape of its output. Prepending two random vectors to the embedding space breaks that lock-in and accuracy rises to 51.6%. Run ten random seeds and take the plurality answer and it reaches 72%, with every task solved by at least one seed. Effects were statistically significant across two models (McNemar p < 0.001), and the code is public.

The cost delta, and the four conditions attached to it

Now the conditions, because this is where a platform decision would go wrong. It requires 8-bit quantization — 4-bit nearly kills the effect. It requires plurality, not majority voting, since majority voting underperforms baseline when per-seed accuracy sits below 50%. It hurts models already scoring above 75% (a DeepSeek-R1-Distill-1.5B fell from 76% to 74.4%). And on tasks beyond the model's actual knowledge, it fabricated legal content across all outputs. The evidence base is 25 arithmetic, 5 planning and 12 legal tasks, with a legal scorer that was non-deterministic on 9 of 12 — and this same group previously retracted a verbosity claim that turned out to be a code bug.

The pattern across cheap-capability claims

This is the third unverified discount on the table. TheSequence covers Poolside's Laguna S2.1, an open-weight 118B mixture-of-experts model that activates only 8B parameters per token with a 1M-token context, and Prentis's Hive-32B, which claims to beat GPT-5.4 and Claude Opus 4.6 on computer-use tasks at roughly a tenth of the cost per task. That claim is vendor-sourced. The convergence is the signal — cost per task is falling from several directions at once — but none of these numbers were produced on your task distribution.

Before approving another frontier-model line item, ask whether the small model already knows the answer and simply cannot say it. That is a $0.009 problem, not a $0.45 one.

So the deliverable is an eval, not an architecture. Here is the asymmetry to sit with. Scaling parameters bought 4 points; changing the inference method bought 40. That is real enough to test, and conditional enough that the guardrail ships before the prototype.