Karpathy's 600-Line Autoresearch Cut a 1.6B Model in Half

◆ DEEP DIVES

1. 01

   Your ML Dependency Tree Is Under Coordinated Attack — and AI Agents Are Making It Worse

   <h3>The Escalation You Can't Ignore</h3><p>What was reported Thursday as a LiteLLM supply chain compromise has escalated into <strong>the most significant coordinated attack campaign to hit ML/AI tooling</strong>. The SANS @RISK bulletin now documents the full TeamPCP timeline: initial access Feb 28, Trivy compromised Mar 19, all 91 Checkmarx tags overwritten in <strong>7 minutes</strong> Mar 23, LiteLLM poisoned on PyPI Mar 24, and Databricks now investigating an alleged breach. The attackers' strategy was surgical: <strong>compromise security tools first</strong>, then move to AI infrastructure, then monetize via ransomware.</p><p>Simultaneously, <strong>six independent CVSS 9.0+ vulnerabilities</strong> hit AI/ML tools in the same window — this is not one incident but a systemic exposure of the AI toolchain:</p><table><thead><tr><th>Tool</th><th>CVE</th><th>CVSS</th><th>Attack Vector</th><th>Your Exposure</th></tr></thead><tbody><tr><td>FastGPT</td><td>CVE-2026-34162</td><td>10.0</td><td>Unauthenticated HTTP proxy</td><td>Any agent platform</td></tr><tr><td>Langflow</td><td>CVE-2026-33309</td><td>9.9</td><td>RCE bypassing prior fix</td><td>LLM workflow builder</td></tr><tr><td>Spring AI</td><td>CVE-2026-22738</td><td>9.8</td><td>SpEL injection in SimpleVectorStore</td><td>Java RAG applications</td></tr><tr><td>CrewAI</td><td>CVE-2026-2275</td><td>9.6</td><td>RCE via Docker fallback</td><td>Multi-agent orchestration</td></tr><tr><td>NVIDIA APEX</td><td>CVE-2025-33244</td><td>9.0</td><td>Unsafe deserialization (PyTorch <2.6)</td><td>Mixed-precision training</td></tr><tr><td>LoLLMs</td><td>CVE-2026-33340</td><td>9.1</td><td>SSRF in proxy endpoint</td><td>LLM web interface</td></tr></tbody></table><hr><h3>AI Agents Amplify the Attack Surface</h3><p>A study of <strong>117,000+ dependency changes</strong> across thousands of GitHub repos found AI coding agents select known-vulnerable versions <strong>50% more often</strong> than humans. Worse: <strong>~20% of AI-recommended packages are hallucinated names</strong>, and 43% of those hallucinations are <em>deterministic</em> — the same fake package name appears consistently across queries. Attackers are registering these names with malicious payloads. One researcher registered a commonly hallucinated name and observed <strong>30,000 downloads</strong> within weeks.</p><blockquote>Your AI coding agent is the fastest, least security-aware developer on your team, and attackers are building exploit chains specifically for its blind spots.</blockquote><p>The <strong>CrewAI vulnerability</strong> is especially treacherous for data scientists: when Docker isn't available — the default in Jupyter notebooks, Colab, and dev setups — CrewAI falls back to SandboxPython with <strong>no real isolation</strong>. Your prototyping environment is running arbitrary AI-generated code with full system access.</p><hr><h3>The Detection Gap</h3><p>Traditional CVE-based scanning (npm audit, pip-audit) has a <strong>267-day average detection lag</strong> and is blind to self-destructing malware — the Axios attacker's code deleted itself after execution, so audit tools returned clean. Vendor disclosure consistently underestimates scope: Checkmarx's own advisory said "older versions deleted" while independent analysis confirmed <strong>all 91 tags overwritten in 7 minutes</strong>. Build incident response assumptions around worst-case scope.</p>

   Action items

   • Upgrade PyTorch to ≥2.6 across all training infrastructure to patch NVIDIA APEX deserialization vulnerability
   • Audit all PyPI dependencies installed between Feb 28 – Mar 27 for LiteLLM, Telnyx. If present, rotate all credentials accessible from that environment
   • Pin all GitHub Actions to full commit SHAs (not tags) across ML pipeline repos this sprint
   • Isolate agent prototyping environments (CrewAI, Langflow) from production credentials and data stores
   • Audit AI-generated dependencies from the last 90 days — verify each package exists in official registries and is on a non-vulnerable version

   Sources:Your PyPI dependencies are under active attack — LiteLLM, NVIDIA APEX, CrewAI all compromised or critically vulnerable · Your AI coding agent picks vulnerable deps 50% more often than you do — and attackers know it · Claude Code's leaked 4-layer context compression stack is your new agent architecture playbook · Your LLM-powered agents have an unsolved attack surface — Willison's 'lethal trifecta' framework · IBM's 3B-param vision model hits 86.4% on Chart2Summary — and your LLM proxy might be compromised · Multi-agent consensus voting cuts manual vuln review to 11%

2. 02

   Sparse MoE Is Eating Dense Models — The Cost-Performance Frontier Just Shifted

   <h3>Holo3: SOTA GUI Automation at 1/10th the Cost</h3><p>H Company's <strong>Holo3</strong> — a sparse MoE built on Qwen3.5 — activates only <strong>10B of 122B parameters</strong> (8.2% ratio) while scoring <strong>78.85% on OSWorld-Verified</strong>, beating both GPT-5.4 and Opus 4.6. The 35B variant (3B active) is <strong>fully open-source under Apache 2.0</strong> on HuggingFace — small enough to run on consumer hardware.</p><p>The "1/10th inference cost" claim follows directly from MoE sparsity math — activating 8% of parameters should yield roughly an order-of-magnitude compute savings — but <strong>real-world cost depends on memory bandwidth, expert routing overhead, and serving infrastructure</strong>. Don't take the number at face value. Benchmark it.</p><p>Independently, <strong>Arcee Trinity-Large-Thinking</strong> (400B total, 13B active, Apache 2.0) ranked #2 on PinchBench behind Opus 4.6 and hit SOTA on Tau2-Airline. A Transformer co-author (Polosukhin) confirms the trend: <em>"Open-source community is preferring MoE for comparable performance with faster inference."</em></p><hr><h3>The Alibaba Pivot: Open→Closed Ratchet</h3><p>While open MoE models gain ground, Alibaba is systematically closing its best work:</p><table><thead><tr><th>Model</th><th>License</th><th>Successor</th><th>Successor License</th></tr></thead><tbody><tr><td>Qwen3.5-Plus</td><td>Open-source</td><td>Qwen3.6-Plus</td><td><strong>Closed/Proprietary</strong></td></tr><tr><td>Qwen3-Omni</td><td>Open-source</td><td>Qwen3.5-Omni</td><td><strong>Closed/Proprietary</strong></td></tr></tbody></table><p>This is a textbook <strong>open-core bait-and-switch</strong>: seed adoption with free weights, build community, then gate the best capabilities behind paid APIs. If you've been fine-tuning Qwen weights, you're now permanently one generation behind the frontier. <em>Meta and Google could execute the same pivot at any time</em> — Alibaba's trajectory is a preview, not an anomaly.</p><hr><h3>The Convergence Signal</h3><p>Three data points converge: MoE is winning open-source deployments, Qwen3.5's architecture is becoming a platform layer (Holo3 built on it, Alibaba released Qwen3.5-Omni on it), and <strong>Liquid AI's 350M-parameter LFM2.5</strong> outperforms models 2x its size on tool use. The efficient frontier is shifting across every model scale.</p><p>Meanwhile, <strong>GLM-5V-Turbo</strong> from Zhipu AI uses joint RL training across <strong>30+ domains simultaneously</strong> to prevent the see-saw effect (improving one capability degrades another). This joint RL methodology is the more interesting intellectual contribution — <em>watch for their technical report on reward function design and convergence behavior</em>.</p><blockquote>Sparse MoE models are rewriting the cost-performance equation for agentic AI: Holo3 matches frontier dense models at 1/10th the cost, and the open-source 35B variant means you can verify this claim on your own hardware today.</blockquote>

   Action items

   • Download and benchmark Holo3-35B-A3B from HuggingFace against your current GUI automation or computer-use pipeline this week
   • Model the inference cost savings of switching dense frontier model calls to sparse MoE alternatives for your top-3 agentic workloads this quarter
   • Audit your dependency on Qwen open-source weights and establish fallback plans using Llama, Mistral, or Gemma families
   • Investigate GLM-5V-Turbo's joint RL training across task families if you're seeing capability regression in multi-task fine-tuning

   Sources:Sparse MoE agents beat GPT-5.4 at 1/10th inference cost · Claude Code's leaked 4-layer context compression stack is your new agent architecture playbook · Your agent pipeline needs these 3 signals · Your agent harness matters more than your model · OpenAI's 4K-freelancer training pipeline & open-weight models at 1/20th frontier cost · Alibaba's open→closed pivot & OpenRouter's $1.3B bet

3. 03

   Autoresearch and Agent Economics: The Optimization Surface Has Moved

   <h3>Autoresearch: Automated Ablation With an Agentic Outer Loop</h3><p>Andrej Karpathy released <strong>autoresearch</strong> — a 600-line Python framework where a human defines the objective and constraints, and an AI agent iterates through hypotheses, implementations, training runs, and evaluations autonomously. In his initial test: <strong>20 genuine improvements</strong> and 11% faster training over two days.</p><p>The more striking result: Shopify CEO Toby Lütke pointed autoresearch at an internal model and ran <strong>37 experiments overnight</strong>. Result: a <strong>0.8B model outscored the 1.6B predecessor by 19%</strong>. Half the parameters. Significantly better. Found while the CEO slept.</p><p><em>Critical unknowns the headlines omit:</em> What evaluation metric does "19%" refer to? Were experiments sequential or parallel? What was the compute cost of 37 overnight runs on a billion-parameter model? How many were regressions vs. improvements? <strong>n=2 demonstrations are suggestive but not evidence of a generalizable methodology.</strong></p><hr><h3>Where This Is Immediately Useful — and Where It's Dangerous</h3><p>If you have <strong>established training pipelines with reliable evaluation metrics</strong> — classification accuracy, NDCG, perplexity — autoresearch is a near-drop-in accelerator. The Shopify result should trigger an immediate audit: if automated search can find a model <strong>half the size that performs 19% better</strong>, the inference cost savings alone justify the experiment compute.</p><p>The extension to knowledge work is where things break. Azeem Azhar's <strong>AutoBeta</strong> replaces objective metrics with LLM judge panels — essentially RLHF without the humans. The known failure modes are well-documented:</p><ul><li><strong>Position bias</strong>: LLM judges prefer outputs presented first</li><li><strong>Verbosity preference</strong>: longer outputs score higher regardless of quality</li><li><strong>Sycophancy collapse</strong>: optimizing against an LLM judge converges toward outputs the judge "likes," not outputs that are good</li></ul><p>Azhar himself admits early experiments <em>"produced outputs that looked fine but lacked measurable improvement signals"</em> — textbook Goodhart's Law.</p><hr><h3>Convergence: Harness > Model, Everywhere</h3><p>Autoresearch converges with three other signals on the same insight: <strong>your optimization surface is the harness, not the model.</strong></p><ul><li>A Transformer co-author (Polosukhin) reports <strong>10x coding task improvement</strong> from changing how a smaller model edits files — no architecture or weight changes</li><li>Dropbox validated <strong>DSPy for production prompt optimization</strong> on Dash's relevance judge — systematic optimization across models, making prompt investment provider-agnostic</li><li>Thinking token redaction causes <strong>measurable quality regression</strong> in complex workflows — your cost-optimization lever may be a quality-destruction lever</li></ul><p>The strategic implication: if a non-ML-engineer can run 37 experiments overnight and beat a model twice the size, <strong>your team's moat is migrating</strong> from "we know how to run good experiments" to "we have proprietary data, calibrated evaluation infrastructure, and deployment pipelines that compound."</p><blockquote>Autoresearch is legitimate for ML tasks with objective metrics; the 0.8B-beats-1.6B result should trigger an immediate audit of your overparameterized production models, but the extension to knowledge work via LLM-as-judge oracles is an unsolved measurement problem, not a solved product.</blockquote>

   Action items

   • Run autoresearch against your most expensive serving model this sprint — measure wall-clock time to reach current best eval score vs. your manual ablation cadence
   • Evaluate DSPy for your most fragile or expensive production prompt — likely an LLM-as-judge, relevance scorer, or classification prompt
   • Profile quality metrics across thinking-depth budgets for your top-5 hardest use cases before adjusting token limits
   • If building LLM-as-judge evaluation pipelines, implement inter-rater reliability checks: measure agreement across judge models, compare to human annotations, track calibration drift — require Cohen's kappa ≥ 0.6

   Sources:Karpathy's 600-line autoresearch loop: a 0.8B model beat 1.6B by 19% overnight · Your agent pipeline needs these 3 signals: thinking-token quality cliffs, DSPy in prod · Your agent harness matters more than your model — 10x coding gains from tooling alone · Your agentic AI costs $72K/year per instance — here's the routing vs. frontier tradeoff math from RSA 2026 · Your API cost model needs updating — Sonnet at 50-65% margins vs Opus at 35-50%