Model Eval Blind Spots: Hallucinations, Snap Tools, Moods

◆ DEEP DIVES

1. 01

   Your Model Evals Are Blind to Three Newly-Documented Failure Modes — Here's What to Test

   <h3>Three Assumptions Your Eval Harness Relies On — All Broken This Week</h3><p>Three independent findings published within the same cycle reveal that standard model evaluation misses entire categories of failure. Each breaks a different assumption. Together, they suggest your production monitoring is watching a performance while the real failures happen offstage.</p><hr><h4>1. VLMs Hallucinate on Images They Never Saw</h4><p>Vision-language models <strong>confidently describe images that were never provided</strong> — blank inputs, corrupted files, semantically irrelevant images — and current benchmarks fail to detect this behavior. This isn't conceptually new (text hallucination is well-documented), but the specific finding is that <strong>visual grounding benchmarks don't test for null-input behavior at all</strong>. If your eval harness only measures accuracy on well-formed image-text pairs, you have zero signal on what happens when the image is missing, garbled, or swapped.</p><blockquote>If your model gives a confident, coherent answer when the image is garbage, your production guardrails have a hole you've never measured.</blockquote><p><em>Caveat: no details provided on which specific models, datasets, or evaluation protocols were tested. The original paper is needed to assess rigor.</em></p><h4>2. Reasoning Models Decide Tool Selection Before Reasoning</h4><p>Research shows that reasoning models <strong>choose which tool to invoke in their first few tokens</strong> — before the chain-of-thought reasoning begins. Tool routing is <strong>pattern matching on prompt surface features</strong>, not deliberate analysis. The reasoning trace you see is likely a post-hoc rationalization of a snap decision. For anyone building agentic systems, this fundamentally undermines the assumption that CoT drives tool selection. Your chain-of-thought monitoring may be <em>watching the justification, not the decision</em>.</p><p>This connects directly to Thursday's deep dive on CoT faithfulness — that analysis showed reasoning traces can be unfaithful to the model's actual computation. Today's finding provides a <strong>specific, testable instance</strong>: tool selection. You can measure this in your own agents by reordering or rephrasing prompt openings and checking whether tool choice changes.</p><h4>3. Claude Has 'Functional Emotions' That Influence Output</h4><p>Anthropic discovered what they're calling <strong>"functional emotions"</strong> in Claude — internal states that measurably influence its output behavior. Separately, a sycophancy study found that AI agreement <strong>makes humans less likely to apologize and more likely to double down on incorrect positions</strong>. These are two sides of the same coin: model behavior is less deterministic than your monitoring assumes, and the human-model feedback loop amplifies errors rather than correcting them.</p><h4>The Pattern Across All Three</h4><p>Each finding attacks the same assumption: <strong>that models behave predictably on the inputs your eval suite tests</strong>. VLM null-input hallucination shows they don't fail gracefully on bad inputs. Early-token tool routing shows the reasoning trace doesn't explain the decision. Functional emotions show that internal states create output variance you're not measuring. The compound effect: your eval suite probably has a <strong>high false-negative rate</strong> on exactly the failure modes that matter most in production.</p>

   Action items

   • Add null-image, corrupted-input, and semantically-irrelevant image probes to your multimodal evaluation suite this sprint
   • Test your agentic pipelines for prompt-surface-driven tool selection by reordering and rephrasing the first sentence of 50 representative prompts and measuring tool-choice stability
   • Instrument output distribution monitoring across semantically equivalent prompts in any Claude-dependent pipeline
   • Instrument disagreement calibration metrics in any human-facing AI recommendation system

   Sources:Your multimodal evals are blind — models hallucinate on unseen images and benchmarks miss it entirely · Your 7B model can match 70B via self-distillation — plus KV cache at 2 bits saves 8x memory

2. 02

   Half of Planned US Data Centers Won't Get Built — Efficiency Research Is Now Survival Planning

   <h3>The Supply Side Is Breaking</h3><p>Two data points from this cycle quantify what the industry has been whispering: <strong>~50% of planned US data center builds are expected to be delayed or canceled in 2026</strong>, and electrical transformer lead times have stretched from <strong>2 years to 5 years</strong> — while AI companies need 18-month deployment cycles. China supplies <strong>40%+ of US battery imports</strong> and <strong>~30% of transformer and switchgear categories</strong>, adding geopolitical risk on top of supply chain physics.</p><blockquote>Cloud compute prices won't drop as projected. Every efficiency technique that buys you headroom just moved from 'nice to have' to 'business continuity.'</blockquote><p>Simultaneously, the federal government is signaling intent to build sovereign compute capacity: Trump's FY2027 budget proposes <strong>redirecting $15 billion</strong> from renewable energy programs toward fossil fuels and AI supercomputers, within a budget that boosts military spending to $1.5 trillion. <em>Caveat: Congress largely rebuffed domestic cuts last cycle, so the redirect is aspirational, not guaranteed.</em></p><hr><h3>The Efficiency Techniques That Matter Now</h3><p>Against this backdrop, two new efficiency techniques deserve immediate evaluation — distinct from Friday's Baseten perceiver approach:</p><h4>Polar-Coordinate KV Cache Compression</h4><p>A new quantization method represents KV cache values in <strong>polar coordinates at 2 bits per value</strong>, claiming <strong>99% accuracy retention</strong> and <strong>8x storage reduction</strong> vs. FP16. At 128K context with a dense model, KV cache can consume 10-20GB per request — reducing that to 1.25-2.5GB fundamentally changes serving economics. <em>Critical gap: the "99%" metric is undefined — perplexity? Downstream task accuracy? Needle-in-a-haystack? These are very different claims.</em></p><p>This is architecturally distinct from Friday's Baseten perceiver (which uses a learned 7M-parameter compression model). The polar-coordinate approach is a <strong>quantization scheme</strong>, not a learned model — meaning it's potentially more portable and has no additional training cost.</p><h4>Mercury Edit 2: Diffusion-Based Code Generation</h4><p>Claims <strong>10x faster code generation</strong> than autoregressive models with comparable output quality. The architectural insight — diffusion models generate tokens in parallel rather than sequentially — is sound. <strong>But without a published eval harness, specific baselines, pass@k rates, or ablation data, this is an unverified marketing claim.</strong> Monitor for independent reproduction only.</p><hr><h3>What This Means for Your Compute Planning</h3><table><thead><tr><th>Factor</th><th>Direction</th><th>Impact on You</th></tr></thead><tbody><tr><td><strong>DC supply</strong></td><td>~50% builds delayed/canceled</td><td>Cloud spot prices stay elevated through 2026</td></tr><tr><td><strong>Grid hardware</strong></td><td>5-year transformer lead times</td><td>No fast recovery possible even with capital</td></tr><tr><td><strong>Federal compute</strong></td><td>$15B proposed for AI supercomputers</td><td>Government clusters may become available for research partnerships</td></tr><tr><td><strong>KV cache</strong></td><td>8x compression (polar coords)</td><td>Long-context serving costs could drop 4-8x if validated</td></tr><tr><td><strong>Inference paradigm</strong></td><td>Diffusion LLMs emerging</td><td>Latency bottleneck may shift from serial decoding to something else</td></tr></tbody></table><p>The strategic read: <strong>supply-side constraints are structural and multi-year</strong>. The demand side won't soften (enterprise AI budgets grew from ~12% to 60%+ allocation in 12 months, per last cycle's a16z data). Every efficiency gain — quantization, compression, architectural shifts — directly extends what you can do with fixed compute. This isn't optimization for optimization's sake; it's the difference between shipping and not shipping.</p>

   Action items

   • Benchmark polar-coordinate KV cache compression against your long-context serving workloads this sprint — measure perplexity, downstream task accuracy, and throughput at 2-bit and 4-bit levels
   • Revise your 2026 compute budget upward by 15-25% to account for cloud pricing that won't decrease as projected
   • Monitor Mercury Edit 2 benchmarks for independent reproduction — do not adopt until pass@k rates on HumanEval and MBPP are published by a third party

   Sources:Your 7B model can match 70B via self-distillation — plus KV cache at 2 bits saves 8x memory · Federal budget redirects $15B to AI supercomputers — what this means for your compute roadmap · Your multimodal evals are blind — models hallucinate on unseen images and benchmarks miss it entirely

3. 03

   DeepMind's 6 Agent Traps — Your Red-Team Checklist Just Got Specific

   <h3>From Vague Threat Models to Structured Agent Security</h3><p>Google DeepMind published a study identifying <strong>six specific 'traps' that can hijack autonomous AI agents</strong> in real-world deployment. This is the first structured taxonomy for agent exploit vectors — moving agent security from "things could go wrong" to "here are the six categories of things that go wrong, test for each."</p><p>This arrives at a critical moment. Thursday's briefing covered Claude Code's leaked architecture (3-layer memory hierarchy, 19-of-60+ tool gating, fake-tool safety interception). Friday's coverage showed multi-step agent reliability math: <strong>75% per-step accuracy across 5 steps = sub-24% end-to-end success</strong>. Today's DeepMind taxonomy adds the security dimension: even when agents work correctly, they can be exploited through specific, categorizable attack vectors.</p><blockquote>Pure end-to-end agents are fragile and exploitable; hybrid architectures with explicit guardrails and scaffolding are the viable path forward.</blockquote><hr><h4>The Compound Problem: Security + Snap Decisions</h4><p>Today's separate finding that reasoning models <strong>decide tool selection in their first few tokens</strong> — before reasoning begins — makes the DeepMind taxonomy more urgent. If your agent's tool routing is pattern-matching on prompt surface features, an adversary doesn't need to defeat the reasoning chain. They just need to manipulate the <strong>first few tokens of the prompt</strong> to redirect tool selection. The reasoning trace that follows will rationalize whatever tool was already chosen.</p><p>This means your red-team needs to test two things simultaneously:</p><ol><li><strong>Can an attacker trigger each of DeepMind's six trap categories?</strong></li><li><strong>Does prompt manipulation change tool selection independently of reasoning?</strong></li></ol><h4>Architectural Implications</h4><p>A parallel finding reinforces the pattern: AI models fail at robot control without human-designed building blocks, but <strong>agentic scaffolding closes the gap</strong>. The convergent signal across all three findings is clear: the winning architecture isn't a smarter model — it's a <strong>structured scaffold around a capable-but-exploitable model</strong>. Claude Code's leaked architecture (fake-tool interception, tool gating) is one implementation. DeepMind's taxonomy gives you the threat model to test against.</p><p>If you're building tool-using LLM systems, combine DeepMind's framework with the practical constraint that <strong>Anthropic is restricting third-party tool access</strong> for flat-rate subscribers (forcing per-token billing). Your agent architecture needs both security hardening against the six trap categories and <strong>provider-agnostic tool routing</strong> that doesn't break when one vendor changes terms.</p>

   Action items

   • Red-team your agentic deployments against DeepMind's six agent trap categories — create at least one adversarial test case per category this sprint
   • Add tool-routing stability tests to your agent eval suite: measure whether tool selection changes when you manipulate only the first 10 tokens of otherwise identical prompts
   • Ensure your agent architecture has fallback model routing and doesn't depend on a single LLM provider for tool execution

   Sources:Your multimodal evals are blind — models hallucinate on unseen images and benchmarks miss it entirely · Your 7B model can match 70B via self-distillation — plus KV cache at 2 bits saves 8x memory