Harness engineering · Essay
The value in enterprise AI has moved out of the sentence you type and into the system around the model. That system is the harness — and it is where the engineering lives now.
Asking a model whether 97 is prime tells you nothing about whether it can do your work. It is prime. The model will say so. Nothing has been learned.
That test belongs to a debate that peaked in 2023. Since then, the value in enterprise AI has moved out of the sentence you type and into the system around the model.
That system is the harness: the tools a model may call, the context it receives, the contract it works under, and the checks it must pass before anything ships.
The prime-number debate is mostly a 2023 story. Its techniques come from 2022 research, it peaked in mid-2023, and the field has renamed itself twice since.
| When | Era | What happened |
|---|---|---|
| Jan 2022 | Prompt craft | Chain-of-thought prompting shows that worked reasoning examples in the prompt lift math scores. |
| Nov 2022 | Prompt craft | ChatGPT launches. Prompt wording becomes the main lever most users have. |
| Jul 2023 | Prompt craft | Chen, Zaharia and Zou report GPT-4's prime-number accuracy falling from 97.6 % in March to 2.4 % in June. |
| Jul 2023 | Prompt craft | Narayanan and Kapoor note the test held only primes. On composite numbers, the June model did far better. |
| Mar 2024 | Prompt craft | The revised paper uses a balanced set: 84 % falls to 51 %, partly because GPT-4 followed chain-of-thought instructions less. |
| Jun 2025 | Context | Tobi Lütke and Andrej Karpathy back context engineering over prompt engineering. |
| Dec 2025 | Context | The Linux Foundation forms the Agentic AI Foundation around MCP, AGENTS.md and goose. |
| Feb 2026 | Harness | Mitchell Hashimoto writes about engineering the harness. OpenAI follows with its harness engineering memo (overview). |
| Feb 2026 | Harness | LangChain lifts its coding agent from 52.8 % to 66.5 % on Terminal-Bench 2.0. The model stayed the same. |
Bringing the prime-number test into a 2026 architecture discussion is like judging a car by revving its engine in the garage.
The prime test fails as an evaluation in four ways, and each one points to the harness.
An agent is a model plus a harness. The model is the engine. The harness turns its output into finished, checked work.
The clearest evidence is LangChain's February 2026 experiment. With gpt-5.2-codex held fixed, changes to prompt, tools and middleware alone added 13.7 points. Their most common failure: the agent wrote a solution, re-read its own code, judged it fine, and stopped.
A production harness has seven parts:
Every mistake an agent makes should end as a permanent change to its harness, not as a better-worded prompt.
If the harness is the product, its outcomes are the metric. DORA's 2025 research, with nearly 5,000 respondents, finds AI acts as an amplifier: it strengthens strong organizations and magnifies the dysfunctions of weak ones. The harness is where you steer that amplification.
| Metric | What it answers |
|---|---|
| Task success rate | Share of tasks that meet the contract's success criterion, checked by tests rather than by the agent |
| Cost per completed task | Tokens, compute and human review minutes per finished task, not per call |
| Change failure rate | How often agent-authored changes cause incidents or rollbacks |
| Rework rate | How often finished work has to be redone |
| Human intervention rate | How often a gate or escalation was needed, and whether that falls over time |
Take a baseline for each metric before the rollout. Adoption is irrelevant if it isn't measurable.
Stop asking whether the model knows that 97 is prime. Ask what share of real tasks it completes inside your harness, at what cost, and how often a human has to step in.
Models will keep improving, and you do not control that curve. You control the harness. That is where the engineering lives now.