Harness engineering means: every time an agent makes a mistake, engineer it out of the system for good. OpenAI used it to ship a million lines with zero hand-written code. Learn the difference between soft and hard constraints, how to make your environment legible to agents, and why lights-off factories still fail.
Key takeaways
- Harness engineering: whenever an agent makes a mistake, build something so it can't happen again.
- Hard constraints (types, lints, CI, architecture rules) beat soft ones (memory files, skills), which can be forgotten.
- Make error messages teach: a failing check should tell the agent how to fix it.
- Make the environment legible: the agent should be able to run, test and observe everything a human can.
- Keep humans steering. Factories where nobody reads the code degrade within months.
Try it with your agent
Mine your history for repeat mistakes:
Look at our last 20 commits and any review comments. Find mistakes that recur. For each, propose the most durable fix: a lint rule, a test, a type, or a hook, with a memory-file line only as a last resort. Implement the top one, with an error message that teaches.
Transcript
Intro
Mitchell Hashimoto describes his whole approach in one sentence. Anytime you find an agent makes a mistake, you take the time to engineer a solution, such that the agent never makes that mistake again. That's harness engineering.
Inner and outer harness
Dex Horthy, following Martin Fowler's site, splits it into two parts. The inner harness is what your agent ships with: its loop, its tools, its system prompt. The outer harness is what you build around it: memory files, skills, commands, hooks, checks, and the way your codebase and dev environment are organized. You can't change the first much. The second is yours.
The OpenAI experiment
How far can this go? OpenAI's Ryan Lopopolo describes a team that shipped a real internal product in five months: about a million lines of code, fifteen hundred pull requests, seven engineers, and zero lines written by hand. Humans steer. Agents execute.
His key observation: building software still demands discipline, but the discipline shows up more in the scaffolding than in the code. The tooling, abstractions and feedback loops that keep the codebase coherent become the real work.
Fix it forever
Here's what fixing a mistake forever looks like. Agents kept calling fetch without a timeout. Instead of correcting it in every review, Lopopolo's team wrote a custom lint rule. And the error message doesn't just say no. It explains the rule and how to fix it, so the agent repairs its own code.
Hard beats soft
Lauren Tan draws a sharp line between soft and hard constraints. Memory files, skills, style guides and review bots are soft. They can be forgotten. Types, lints, CI checks and enforced architecture are hard. If you only have soft constraints, she warns, it's only a matter of time before your codebase looks like complete trash.
Her team goes further. Everything you can imagine the agents are bad at, she says, we ban. A React hook agents misuse, banned. Rambling code comments, banned. Imports that cross an architecture boundary, blocked by a check.
Legible environments
The other half of the harness is the environment. The Claude Code team makes sure the agent can do everything a human does to build and test the product end to end. OpenAI teaches its agent to launch the app, start the observability stack and drive a browser. And Karpathy's pet peeve: docs that say go click this URL, instead of giving you text to paste to your agent.
Factories fail
But don't remove the humans. Dex Horthy's team built a lights-off software factory in July 2025, where nobody read the code. By November, they had shut it down. After three to six months without human steering, the codebase was easier to restart than to fix.
Keep it light
And hold your harness loosely. Boris Cherny notes that as models get better, the harness matters less. The Claude Code team builds scaffolding knowing it will be deleted, like the to-do list tool they needed for older models. Build what today's models need, and prune as they improve.
Quick check
Quick check. Agents keep calling fetch without a timeout, and reviewers keep catching it. What's the most durable fix? Tap your answer.
Try it
Start your harness engineering habit. This prompt finds recurring mistakes in your history and turns the worst one into a durable check.
Recap
Recap. Every mistake is a chance to fix the system, not just the code. Prefer hard constraints, with errors that teach. Make your environment legible to agents. Keep humans steering, and keep the harness light. That's module four. Next: codebases agents can work in.