Research · Area 01 of 07
Harness Engineering
An agent is a model plus a harness, and the harness gap is measured, not anecdotal: independent 2026 studies found the same model swinging more than ten points on real tasks purely from harness choice.
Stoa Labs studies how scaffolding changes what an agent can do and how far it can be trusted: the design of agent harnesses, the outer loops that run them, the test harnesses that make non-deterministic systems checkable, and the verifiers and stop rules where reliability actually lives.
Every harness component encodes an assumption that the model cannot do something on its own, and those assumptions expire as models improve, so we treat harness engineering as a moving research target rather than a settled architecture.
01 · Definition
A model plus a harness
The working equation of the field is Agent = Model + Harness. The harness is everything around the model that turns raw capability into completed work: the deterministic software that manages tools, context, memory, permissions, retries, and verification so the model can focus on reasoning. By mid-2026 harness engineering is an established term used by frontier labs and practitioners, with an academic survey under review, not just blog vocabulary.
Two canonical decompositions describe what a harness contains. LangChain’s anatomy names five primitives: filesystem as durable state, code execution, sandboxing, memory, and context management. The broader community taxonomy expands this to a twelve-category engineering checklist covering the agent loop, planning, tool design, permissions, verification, observability, and human-in-the-loop.
A framing we adopt as central: every harness component encodes an assumption that the model cannot do something on its own, and those assumptions expire as models improve. A harness is scaffolding you are always partly tearing down. This is why harness engineering is a moving target rather than a settled architecture, and why it rewards continuous research over one-time design.
02 · Why it matters
Where the differentiation moved
- Model convergence pushes differentiation into the harness. Frontier models inside coding agents have largely converged on capability, so the scaffolding now does most of the differentiating work. Value is migrating to exactly the layer where a specialist can compete without training a model.
- The harness gap is measured, not anecdotal. Two independent 2026 studies (Tongyi Lab’s PawBench, 4,050 runs across 9 models and 3 harnesses, and Harness-Bench) show the same model swinging more than 11 points on real tasks purely from harness choice, with weaker models far more sensitive than strong ones. A weaker model on a better harness beat a stronger model on a worse one: the harness gap is comparable to a model upgrade.
- The bottleneck moved from prompts to loops. As agents became reliable at long-horizon work, the scarce skill stopped being prompt writing and became designing the system that prompts the agent.
- Fleets create a coordination and governance gap. Teams running several harnesses at once hit siloed sessions, non-portable guardrails, and cost opacity, which is the gap the meta-harness layer targets.
- Non-determinism breaks conventional testing. Agents do not run the same way twice, creating demand for deterministic replay, simulation environments, and state-based verification.
03 · Sub-areas
Five sub-areas, one discipline
- Agent harnesses. The design and engineering of the harnesses themselves: context delivery, tool surfaces, permission models, verification gates, and the trade-offs among cost, quality, and speed.
- Meta-harnesses. The term carries two distinct meanings, and we work with both. Horizontal: an orchestration layer above many harnesses that composes, governs, and shares them, a harness of harnesses. Vertical: systems that automatically optimize a single harness, where an outer-loop meta-agent rewrites the harness code around a fixed model. We treat the vertical meaning as the research-rich one.
- Test harnesses. How agents and harnesses are tested: deterministic record-and-replay for debugging and golden-trace regression, simulation environments with LLM-simulated users and tool worlds, and sandbox-based verification that grades final world state rather than conversation text.
- Loop engineering. Designing the system that prompts the agent rather than the prompts themselves. The inner loop is the agent’s own plan, act, verify cycle; the outer loop is the slower system wrapping many inner cycles: triggers, verifiable goals, verification, and memory. Verifier and stop-rule design is where reliability actually lives, because the recurring failure is agents declaring victory without checking.
- Harness evaluation. Measuring harness effects directly: harness-sensitivity studies, and benchmarks that evaluate the model and harness together.
04 · Open questions
What we are trying to find out
Exploration-mode questions: pursued to learn something, allowed to not pan out.
- Can vertical harness optimization be made cheap and transferable? The published method works but is compute-hungry and benchmark-specific; discovering harnesses that survive a model swap is open.
- What explains harness sensitivity, and can model-harness fit be predicted from measurable properties (tool count, validation gates, context policy) rather than brute-force evaluation?
- Can verifier and stop-rule design measurably reduce victory-declaration bias and the other named failure modes, context anxiety and one-shotting overreach among them?
- How far can deterministic replay and golden-trace regression substitute for LLM-simulated users, given the evidence that simulated users are unreliable proxies for humans?
- Which harness assumptions expire next as models improve, and what does a harness that sheds scaffolding gracefully look like?
05 · Directions
What this work could become
Candidate development directions, not commitments or year-one programs.
- Harness optimization tooling. Use Stoa-owned, public, licensed, or independently synthetic tasks and traces to discover better harnesses: the unclaimed engineering between the vertical meta-harness papers and a usable product.
- A harness-sensitivity dataset and model-harness fit advisory. There is no neutral, trusted source for which harness should run which model, and why. The dataset is built independently from permitted research inputs; client measurements remain inside their engagements.
- A reliability harness. Deterministic replay, golden-trace regression testing, and a verifier library packaged for teams that need agents they can trust.
- Reference outer-loop patterns. Loop engineering is weeks old as a named discipline; authoring its reference patterns is positioning and pipeline value, with reusable components as the byproduct.
06 · Boundaries
What we deliberately avoid
Building a horizontal meta-harness or governance gateway. That space filled in a single week of June 2026, it competes on capital and distribution, and its runtime primitives are being commoditized by open standards. Our work sits on the method, measurement, and verification seam instead.
07 · Landscape · 2026-07
The landscape, July 2026
The major harnesses are Claude Code, Codex CLI, Cursor, and Google’s Antigravity CLI, with Pi and OpenCode as the leading open harnesses. Pi is becoming a foundational substrate that others build on, including Cloudflare’s Flue framework.
June 2026 saw a dense burst of activity across the whole layer above the single harness: Databricks open-sourced Omnigent, the reference horizontal meta-harness, TrueFoundry launched its governance gateway, Vercel shipped HarnessAgent in AI SDK 7, and Zed’s Agent Client Protocol registry now connects more than 25 agents to editors. A precision note: ACP is a protocol and Flue is a framework built on Pi; neither is a meta-harness, so the accurate story is one burst across protocol, framework, composition, meta-harness, and governance layers.
On the research side, Stanford’s Meta-Harness showed an outer-loop agent rewriting harness code can beat hand-engineered baselines on Terminal-Bench, and the PawBench and Harness-Bench studies quantified harness sensitivity. Loop engineering crystallized as a named discipline in June 2026. Corporate volatility is real: SpaceX agreed to acquire Cursor for $60B in June 2026.
Landscape notes are dated on purpose. This layer moves monthly, and we treat any claim older than a quarter as due for reverification.
08 · Sources
Selected sources
- OpenAI, "Harness engineering: leveraging Codex in an agent-first world"
- Martin Fowler, "Harness engineering for coding agent users" (April 2026)
- Lee et al., "Meta-Harness: End-to-End Optimization of Model Harnesses" (arXiv 2603.28052)
- Tongyi Lab, "The Harness Gap" / PawBench v1.0 (June 2026); "Harness-Bench" (arXiv 2605.27922)
- Databricks, "Introducing Omnigent" (June 2026)
- Terminal-Bench 2.0 and Harbor (tbench.ai)
Where this work goes
This work supplies the methods behind our harness and orchestration engineering services, and its findings are published with their methods and limits as the evidence allows.