Harness Engineering Emerges as the AI Coding Unlock

◆ DEEP DIVES

1. 01

   Harness Engineering Is Here: The Concrete Playbook from OpenAI, Stripe, and Anthropic

   <h3>The Environment Was Always the Bottleneck</h3><p>Mitchell Hashimoto coined the term <strong>harness engineering</strong> — the practice of building constraints, tools, documentation, and feedback loops that keep coding agents productive. The key insight, validated independently across OpenAI, Stripe, and Anthropic: <strong>agent capability was never the constraint</strong>. The environment was. Think Docker for AI-generated code — it didn't make applications faster, it made deployment reliable by constraining the environment.</p><blockquote>OpenAI's 3-person team built a million-line internal product in five months with zero hand-written code, averaging 3.5 PRs per engineer per day. Their secret wasn't a better model — it was a strict layered architecture with rigid dependency boundaries enforced by custom linters.</blockquote><h3>The Patterns That Work</h3><p>The convergence across organizations is striking. Here are the concrete, adoptable patterns:</p><ul><li><strong>AGENTS.md as a living feedback loop</strong>: Not static documentation — every time an agent makes a mistake, you add a line preventing that class of error permanently. Ghostty's AGENTS.md has each line corresponding to a specific past agent failure. OpenAI uses a hierarchical approach: a small AGENTS.md pointing to deeper design docs, architecture maps, and quality grades, all versioned in the repo.</li><li><strong>Custom linter rules with remediation instructions</strong>: When an agent violates an architectural constraint, the error message tells it exactly how to fix the violation. This creates a self-correcting loop that doesn't require human intervention for known failure modes. The linters themselves were Codex-generated.</li><li><strong>MCP-exposed internal tooling</strong>: Stripe's Toolshed platform exposes <strong>400+ internal tools via MCP servers</strong>, giving agents the same operational surface area as human engineers.</li><li><strong>JSON over Markdown</strong> for agent-facing structured data: Anthropic discovered agents treat Markdown as prose they can freely rewrite, but respect JSON's structure. Small detail, big implications for agent-facing configuration.</li><li><strong>Plan-then-execute as mandatory workflow</strong>: No agent writes code until a human has reviewed and approved a written plan.</li></ul><h3>The Honest Failure Modes</h3><p>Agent-generated code <strong>accumulates entropy differently</strong> than human-written code. OpenAI runs periodic 'garbage collection' agents but admits it's an emerging practice. Anthropic found agents marking features as complete without proper end-to-end testing — their browser automation tools have blind spots (<em>Puppeteer can't see native alert modals</em>). The human review bottleneck is real: practitioners cap out at <strong>3-4 parallel agent sessions</strong> before becoming the constraint.</p><blockquote>Every success story is greenfield. Applying harness engineering to a legacy codebase with inconsistent testing, implicit conventions, and patchy documentation is an open problem nobody has convincingly solved.</blockquote><h3>Why This Compounds</h3><p>Every AGENTS.md update, every custom linter rule, every MCP-exposed tool accelerates all future agent work. Stripe's unattended model — developer posts a task in Slack, agent writes code in a pre-warmed sandboxed devbox, passes CI, opens a PR — produces <strong>1,000+ merged PRs per week</strong>. But it requires mature infrastructure most teams don't have yet. The role of the engineer is bifurcating: you're either building the environment or managing the work. Start now, even incrementally.</p>

   Action items

   • Create an AGENTS.md at the root of your primary repositories by end of this week — start with architectural constraints, common pitfalls, and testing conventions
   • Audit your linter rules and add agent-friendly remediation instructions to error messages within this sprint, prioritizing architectural boundary violations
   • Designate an 'agents captain' on each team by end of month — someone responsible for evaluating agent fit, maintaining the harness, and championing adoption
   • Inventory your internal tools and create a prioritized list for MCP server exposure this quarter, starting with most-used CLI tools, deployment scripts, and monitoring dashboards

   Sources:The Emerging "Harness Engineering" Playbook · 🧠 Intelligence should be owned, not rented · Welcome to the free edition of The Pragmatic Engineer Newsletter

2. 02

   MCP Is the New Microservices Security Problem — and Attackers Are Already There

   <h3>The Protocol Layer Is Under Active Attack</h3><p>Here's the tension you need to internalize: the same MCP protocol that harness engineering depends on for tool exposure is <strong>already being actively probed by attackers</strong>. Cisco's SVP of AI Software and Platform confirms it plainly — agents are being hijacked, impersonated, and manipulated to exfiltrate data or execute unauthorized commands <strong>'at machine speed.'</strong></p><blockquote>MCP and agent-to-agent protocols scaled as connectivity standards, not security standards. They tell agents how to talk to tools and each other, but the identity, authorization, and behavioral monitoring layers are bolted on after the fact, if at all.</blockquote><h3>The Microservices Parallel Is Exact</h3><p>If you lived through the early microservices era, you've seen this movie. Everyone was excited about service decomposition, but the real production pain was <strong>service-to-service auth, mTLS, and observability</strong>. We're in that exact same phase with agentic AI. The prescription maps cleanly:</p><table><thead><tr><th>Microservices Pattern</th><th>Agent Equivalent</th></tr></thead><tbody><tr><td>Service identity (mTLS)</td><td>Zero-trust identity for agents as first-class IAM principals</td></tr><tr><td>API gateway / service mesh</td><td>Controlled tool registries — agents access only explicitly granted capabilities</td></tr><tr><td>Distributed tracing</td><td>Continuous behavioral monitoring — detecting action pattern deviation</td></tr><tr><td>Circuit breakers</td><td>Human-in-the-loop gates for privileged operations</td></tr></tbody></table><h3>Where to Draw the Human-in-the-Loop Line</h3><p>The design constraint is crisp: anything that affects <strong>trust, access, or control over critical systems</strong> — granting privileges, changing production environments, authorizing sensitive data access, initiating irreversible actions — should never run fully autonomous. The 80/20 heuristic for autonomous incident resolution gives you a planning model: design for full autonomy on the <strong>pattern-matching 80%</strong>, and design explicit escalation paths for the complex 20%.</p><p>This directly intersects with harness engineering. If you're exposing 400 tools via MCP like Stripe, each tool needs an explicit authorization model. <em>The speed at which you adopt agent tooling must not outpace the speed at which you secure it.</em></p>

   Action items

   • Audit all MCP and agent-to-agent protocol implementations for identity, authorization, and behavioral monitoring gaps by end of this sprint
   • Map your agent capabilities against a privileged-operations checklist (production deployments, PII access, privilege grants) and add human approval gates for each by end of month
   • Evaluate adding agent identity as first-class principals in your IAM system this quarter

   Sources:🧠 Intelligence should be owned, not rented · The Emerging "Harness Engineering" Playbook

3. 03

   Your LLM Abstraction Layer Needs Model Routing — Here's the Evidence

   <h3>Multi-Model Is Now Default, Not Experimental</h3><p>Three independent sources this cycle confirm the same pattern: <strong>no single model wins across all tasks</strong>, and the teams getting the most value are routing tasks to the right model. Even GPT-5 power users still reach for <strong>Claude for coding</strong> and <strong>Gemini for long-context analysis</strong>. This isn't benchmark data — it's consistent practitioner behavior across different organizations and use cases.</p><h3>GPT-5's Platform Play Changes the Integration Calculus</h3><p>GPT-5 now ships with <strong>native platform connectors</strong> (e.g., HubSpot), signaling OpenAI is moving from model provider to integration platform. You now have three competing patterns for LLM-powered tool integration:</p><ol><li><strong>Custom function-calling</strong>: You define tools and handle API calls yourself. Maximum control, maximum maintenance.</li><li><strong>MCP or similar open protocols</strong>: Tool discovery and execution via an open standard. Portable but security-immature (see above).</li><li><strong>OpenAI native connectors</strong>: Fastest to ship, hardest to migrate away from. Vendor lock-in by design.</li></ol><p>If you're hardcoding <code>model='gpt-5'</code> throughout your codebase, you're accumulating tech debt. A routing layer that selects models based on task classification pays for itself quickly — and becomes essential as the model landscape continues to shift quarterly.</p><h3>The Two-Pass Pattern for User-Facing Features</h3><p>The 'meta prompting' technique — asking an LLM to rewrite a prompt before executing it — is the prompt-engineering equivalent of <strong>query rewriting in search/RAG systems</strong>. The interesting UX detail: generating multiple-choice clarification options instead of free-text follow-ups. This structured disambiguation reduces cognitive load on users while constraining the input space, making the refinement step more reliable. If you're building user-facing AI features where prompt quality varies wildly, this two-pass pattern with structured output on the first pass is worth implementing.</p>

   Action items

   • Add model routing to your LLM abstraction layer this quarter if you don't already have it — treat the model as a swappable dependency, not a hardcoded constant
   • Evaluate GPT-5's native connector architecture within the next two sprints if you're building SaaS integrations through LLMs
   • Prototype a two-pass prompt refinement pattern for one user-facing AI feature this quarter

   Sources:🧠 Intelligence should be owned, not rented · 🤖 Meta Prompting: The Secret to Better AI Results · Welcome Email 2/3: Our Most Popular Issue

4. 04

   Developer Productivity Measurement: LinkedIn's DPH Framework and the LLM Usage Gap

   <h3>LinkedIn Open-Sources Their Productivity Framework</h3><p>LinkedIn has released their internal <strong>Developer Productivity & Happiness (DPH) Framework</strong> — a structured system of metrics, processes, and feedback loops for understanding developer needs. This isn't just a dashboard template; it's an operational playbook covering systems, processes, metrics, and feedback systems. The interesting question is how their metric choices compare to the <strong>DORA/SPACE frameworks</strong> that have dominated the DevEx conversation.</p><p><em>The cargo-culting risk is real</em>: LinkedIn has thousands of engineers, and their measurement overhead makes sense at that scale. For a 20-person team, you need to ruthlessly simplify. But as a baseline taxonomy for what to measure and why, it's the most concrete reference implementation available.</p><h3>The LLM Usage Gap Is Widening</h3><p>A staff engineer's observation resonates with the harness engineering data: engineers getting the most value from LLMs aren't using them for the obvious use case (generate this function). They're using them for <strong>codebase exploration</strong> ('explain this legacy module's error handling patterns'), <strong>RFC drafting</strong> ('here's my architecture decision, argue against it'), and <strong>test generation</strong>. At the staff level, work is less about writing code and more about understanding systems, communicating decisions, and reviewing others' work — all areas where LLMs are surprisingly effective.</p><blockquote>If your team hasn't had a structured conversation about LLM workflows, you're leaving productivity on the table — and the gap between effective and ineffective users is widening every month.</blockquote><p>This connects directly to harness engineering: the teams that formalize how agents and LLMs fit into their workflows — with AGENTS.md, designated champions, and shared patterns — will compound their advantage over teams where adoption is ad hoc and individual.</p>

   Action items

   • Pull LinkedIn's DPH Framework repo and evaluate their metric taxonomy against your current developer productivity measurements within the next month
   • Run a team retrospective on LLM tool usage patterns within the next two sprints — identify where AI tools are working, where they're not, and share effective workflows

   Sources:Welcome Email 2/3: Our Most Popular Issue · The Emerging "Harness Engineering" Playbook