Claude Values Drift by Language Across 309K Chats

The mechanism sits where you'd predict from the training data. RLHF and safety fine-tuning corpora are overwhelmingly English, and refusal/deference is the most fine-tuning-sensitive surface a model has. Anthropic's analysis of 309,815 conversations found Claude's values sorting by locale across four axes: deference vs. caution, warmth vs. rigor, depth vs. brevity, candor vs. execution. Arabic skewed deferential and concise. English skewed toward caution and detail. Dutch was most candid. Indonesian was most task-completing.

Caveat, and it matters: this is observational. Language correlates with topic, intent, and culture, and no effect sizes were published, so part of the signal is query-distribution confounding rather than model behavior. The operational conclusion survives that caveat. Your English regression suite does not generalize to non-English traffic. A frontier lab measured non-invariant safety behavior in its own model. Assuming yours is invariant is now the position without evidence behind it.

The thing an aggregate score doesn't tell you is where the failure lives. A single number averages away a locale where refusal rate collapsed or verbosity doubled. It reads clean while one cohort is broken. This is feature drift on a minority segment, invisible until a user or a regulator finds it for you.

The move

Treat cross-language behavior as a monitored distribution shift with locale-specific acceptance thresholds, not a single pass/fail. Build golden sets per production language and track the four axes as separate metrics.

Start with the counterintuitive receipt. Cognition swapped Opus 4.8 for Fable 5 in Devin and the total bill dropped even though Fable costs 2x per token. Reasoning efficiency, mostly fewer turns and retries, swamped the sticker price. That is the tell: any $/1M-token benchmark is measuring the wrong variable for agentic workloads.

Independent data lands in the same place. Vercel's gateway shows open-weight models handling 29% of token volume at under 4% of spend, roughly a 7x cost-per-token gap, which turns difficulty-based routing (extraction, classification, reranking to open weights) into structural savings rather than a tuning knob. And the assumption that a token is a token does not survive contact with tokenizers. Identical TypeScript costs 73% more on Claude than GPT, and Anthropic's newer tokenizer is about 30% less efficient than its prior versions. On code-heavy traffic that is a material error in any rate-card cost model.

The hardware floor is fragmenting underneath all of this. Capital is moving into inference silicon: FuriosaAI, d-Matrix (explicitly GPU-complementary), SambaNova at $11B, Groq inside Nvidia's $20B deal. None of them publish tokens/sec on your workload, which is the only number that would settle it. Near-term inference is heterogeneous, so a CUDA-locked serving path is a growing liability.

There is one unhedged risk sitting under the cheap-inference story. Deepseek and Qwen carry 25% of OpenRouter usage, and Beijing is weighing export controls on those exact open weights. A cost model that quietly depends on them is a single point of failure with a geopolitical trigger, and most people modeling it have not priced that in.

The move

The correct benchmark unit is cost-per-completed-task under your real harness, not price per token. A provider abstraction earns its keep here for a mundane reason: it makes a backend swap a config change instead of a migration, which is what you want when the cheapest backend keeps moving.