The Harness Is the Product

When an AI interaction disappoints, the reflex is to reach for a bigger model. Sometimes that is the right move. More often the model was never the constraint. The quality of what a language model produces inside a business is the product of at least five distinct components, and the one most organizations control least deliberately is the one with the most leverage.

What Actually Determines the Quality of an AI Interaction

The model, and how hard you let it think

There are two dimensions here, and they get conflated constantly.

The first is model class. A small, fast model and a frontier model are different instruments, not better and worse versions of the same one. The second dimension sits inside a single model: how much effort it is permitted to spend. Reasoning budget, extended thinking, number of passes, whether it is allowed to check its own work. The same weights at low effort and high effort behave like different systems.

Consider two tasks in the same company. Sorting four thousand inbound support emails by urgency has a narrow answer space and a high volume. A small model at low effort is correct here, because cost and latency dominate and the judgment required is shallow. Now take a gap analysis of sixty pages of internal policy against NIST 800-171. That work is cross-referencing, ambiguity resolution, and a judgment about whether stated evidence is actually sufficient. It needs a frontier model with room to think.

Run the second task on the first configuration and you get a confident, well-formatted, wrong document, which is worse than no document. Run the first task on the second configuration and you get a correct answer at fifty times the cost. Model selection is a per-task decision. Most organizations make it once, at the account level, and never revisit it.

Connectors: giving the model your truth

A model with no access to your systems reasons from general knowledge and whatever someone pasted into the window. Connectors change that by making your systems of record readable.

Ask an assistant whether you are at risk on a major client renewal. Without connectors, you get a reasonable framework for thinking about renewal risk. With connectors into the CRM, the ticketing system, and time tracking, you get an answer that names the contract end date, reports that eighty-one percent of the contracted hours are consumed with four months remaining, lists three tickets sitting past SLA, and notes that the executive sponsor has not been on a call in nine weeks. Same model. Entirely different answer, because one of them had the facts.

There is a second, less obvious function. Connectors force a decision about what is authoritative. When the CRM and the finance system disagree about contract value, something has to rule. That is not a technical question, and answering it is one of the more valuable side effects of doing this work properly.

Tool calling: turning answers into actions

Connectors are mostly about reading. Tools are about doing, and they come in two useful flavors.

The first is computation. A model asked to reconcile two ledgers will produce plausible arithmetic, and plausible arithmetic is a liability. Given a code execution tool, it writes and runs actual code, so the numbers are computed rather than predicted, and the twelve variances it surfaces are real.

The second is action in other systems. The model does not tell you to open a remediation task, it opens one, assigns it, links the evidence, and sets the due date. This is where capability stops being the interesting question and permissioning starts. A tool that can write into your accounting system is a very different object from one that can only read it, and the harness is what decides which one the model is holding.

Skills: your judgment, made reusable

Skills are procedural knowledge. They encode how your organization does a specific kind of work, at a level of specificity the model could not possibly infer.

A security assessment readout is a good example. Without a skill, a capable model produces a competent, generic report. With a skill that specifies the section order, the risk ranking method you actually use, the evidence standard for calling a control satisfied, the phrases your legal review will not approve, and the rule that remediation is sequenced by dependency rather than severity, you get something your delivery team recognizes as belonging to the firm.

The strategic point about skills is not the time saved on any single deliverable. It is that a skill takes expertise currently living in the heads of three senior people and makes it available to everyone, consistently, on the first draft.

The harness: the thing that runs everything else

The harness is the layer that decides what the model sees, in what order, which tools are available for this particular piece of work, under what limits, with what verification, and how and when the human participates. It is simultaneously the orchestration logic and the user experience.

Hold the model, the connectors, and the skills constant, and compare two harnesses.

The first is a single chat box. No memory between sessions, so context is retyped every time. Every tool is available for every request. Output is text the user copies somewhere else. Nothing is verified. Nothing is recorded.

The second loads project context automatically on every run, scopes tool access to the task at hand, decomposes long work into subagents with a defined budget, checks results against a test before surfacing them, holds any action with side effects for human approval, and keeps a full trace of what happened. It also knows what it has done before.

The second harness will complete categories of work the first cannot begin. None of that difference is intelligence.

Why the Harness Is the Underrated Variable

Y Combinator devoted a recent Paper Club session to exactly this question, under the title "Why The Harness Matters More Than The Model." The session's framing is that harnesses get written off as mere scaffolding or prompt engineering rather than treated as real research, and that this dismissal is badly wrong. The evidence they lead with is a single comparison: the same model weights that score 30% on ARC-AGI score 95% under a better harness.

That is not a marginal improvement. It is the difference between a system that fails a reasoning benchmark and one that passes it, with no change to the model at all.

Three presentations filled in the argument.

Prime Agent, presented by Seth Karten of Prime Intellect, is an open-source, self-improving harness. It raises ARC-AGI-3 performance from 30% to 95.5%, exceeding the reported human expert baseline, and matches or beats native harnesses across long-context coding, GPU kernel generation, and emulator construction. It sustained an eighty-five hour autonomous nanoGPT run producing nineteen validated records. Two design ideas do the work. Context is treated as a variable the model can programmatically manipulate rather than a transcript it passively accumulates, an argument the talk frames as the shift from a Turing machine to a von Neumann computer, with context managed like L1, L2, and L3 cache. And a persistent harness state stores prompts, memories, executable skills, and subagent specifications as typed, versioned objects that survive across runs, so the system improves rather than starting fresh. Notably, no frontier model has been trained around Prime Agent. The gains come from the harness alone. The paper's own framing of the goal is worth sitting with: the harness exists so that harness failures do not get mistaken for model failures.

OpenJarvis, presented by Jon Saad-Falcon of Stanford, produced the finding with the sharpest business implication. Swapping a frontier cloud model for a capable local model inside an existing personal AI stack drops accuracy by 25 to 39 percentage points. The reason is that the stack, its prompts, its tool descriptions, its memory configuration, its runtime settings, was built around the specific model it shipped with. Tuning the prompts alone recovers only about five points of that gap. OpenJarvis instead decomposes the stack into five independently editable primitives (Intelligence, Engine, Agents, Tools and Memory, and Learning) and optimizes across all of them. Doing so closes the gap to within 3.2 points on average, matching or exceeding cloud accuracy on four of eight benchmarks, while reducing marginal API cost roughly 800-fold and latency fourfold.

QM, presented by Josh France and Regan Bell, is the harness Y Combinator built for its own staff after running more than fifty individual agents internally and hitting the ceiling of what personal-assistant-shaped tooling can do at organizational scale. YC runs it across accounting, legal, events, and engineering, including the work of building QM itself. One of its power users described the effect as a ridiculously lean team that "output like an army."

QM's architecture is instructive for anyone thinking about this in a regulated environment. The unit is the scope rather than the user: every person and every channel gets its own memory, files, credentials, permissions, and sandbox. It is deliberately model-agnostic, with several coding engines interchangeable behind it. Security runs through three named postures. And the detail that says the most about what a production harness actually consists of is the code distribution. Thirteen files implement the model-calling loop. Twenty-six implement access control, identity, authentication, audit, policy, credentials, and security. Twice as much code governs who may see what as drives the model.

The session closes on an observation that lands harder than it first sounds: agents do not understand social context. They do not know which message is politically sensitive, which stakeholder needs to hear something first, or when the correct answer is one nobody should send. That is not a capability gap a larger model closes. It is a harness responsibility, and it will remain one.

Sourcing note: figures above are drawn from the YC session, and the primary papers and releases presented in it (Prime Agent, arXiv 2608.23552; OpenJarvis, arXiv 2605.17172; QM, qm.ycombinator.com). 

The Strategic Difference

Picture two organizations of similar size, in the same market, spending similar money on AI.

Organization A bought licenses for a capable frontier model. It connected email and file storage because those were one-click. It wrote no skills. It uses the default interface as shipped. Adoption looks healthy on the dashboard. Usage is high.

What Organization A actually has is a few hundred people independently discovering how to prompt. Output quality tracks individual talent rather than organizational standard. Nothing accumulates, because every insight about how to make the tool useful dies in the chat window where it was earned. There is no audit trail for work that touched client data. No one can say whether the tool improved a process, because no one defined the process or its success measure beforehand. And when a substantially better model ships, Organization A gets a modestly better chat experience and nothing else, because there is no structure for a better model to lift.

Organization B did unglamorous work first. It picked its highest-volume, judgment-bearing processes and wrote down how they actually run today, including the parts that only work because a specific person notices something. It defined the outcome each process is supposed to produce and what would count as success. Then it built accordingly: connectors to the systems that hold the truth, with an explicit ruling on which system wins a disagreement; tools scoped so that reading is broad and writing is narrow and approved; skills encoding the firm's method, not the model's guess at it; and a harness that maintains context, budgets effort, verifies before surfacing, holds side effects for approval, and keeps a record.

Organization B's advantage compounds along several axes at once.

Why does this matter?

Every component becomes tunable. The OpenJarvis result is the cautionary tale here. A stack implicitly built around one model regresses badly when a component is swapped, and prompt tweaking recovers almost none of it. An organization that has not decomposed its stack cannot optimize it, and cannot even tell which part is failing. Organization B can move to a cheaper model for a high-volume task, or a stronger one with a bigger thinking budget for an analytical task, and measure the result. Organization A can only change vendors.

Evaluation becomes possible. This is the quiet one. Organization B defined what success looks like before deploying, so it can distinguish improvement from noise. Organization A cannot, which means it cannot justify further investment, cannot detect degradation, and cannot tell a genuine capability gain from a lucky prompt.

The asset survives model turnover. Frontier models are replaced every few months. Process definitions, connector topology, authority rules, encoded firm judgment, guardrails, and approval flows are not. Tuning prompts for the current model is renting. Building the harness is owning something that carries forward, and that gets better every time the model underneath it improves. Organization B experiences a model release as a free upgrade across every workflow. Organization A experiences it as a news item.

Governance stops being a bolt-on. The QM code ratio, twice as much code governing access as driving the model, is not an accident of that project. It is what a harness intended for real organizational work looks like. For anyone operating under CMMC, NIST 800-171, HIPAA, or a client contract with data handling terms, that ratio is the strategy. Access scoping, credential isolation, audit trails, and human approval gates are properties of the harness. They cannot be prompted into existence, and adding them later means rebuilding.

Delegation becomes real. The interaction model determines what can be handed off. A chat box supports assistance: a person stays in the loop for every step and remains the bottleneck. A harness with persistent context, verification, and approval gates supports delegation: work runs, gets checked, and arrives for review. The economic difference between assistance and delegation is not incremental, and it is a design decision, not a model capability.

The uncomfortable part of all this is that the hard work is not technical. Defining current-state processes, naming desired outcomes, agreeing on what success looks like, deciding which system of record is authoritative, and choosing where a human must stay in the loop are organizational tasks. They require people to articulate things they have never had to write down. Most AI initiatives skip them, because buying licenses is fast and process definition is slow and political.

But a harness is a codified decision about how work should happen. You cannot encode a decision you have not made. Organizations that treat this as a procurement exercise end up with a very good model producing generic output against undefined goals. Organizations that treat it as an operating design exercise end up with something that is genuinely theirs, and that gets more valuable with every model release.

The model is the part you buy. Everything else is the parts you build.