Agent harnesses

What a harness is

A harness is the software that runs an agent. It assembles what the model sees, asks it what to do, checks whether the proposed action is permitted, executes it, captures the result, decides whether the task is done, and records enough that someone can reconstruct the sequence later.

Everything between a model deciding something and the world changing happens here. That is a larger surface than it sounds, and it is where I have spent most of my building time.

The loop is the easy part

Written out, an agent loop is short. Assemble context. Ask the model. If it proposes a tool call, check it, run it, capture the result. Add the result to context. Repeat until done, limited, or denied.

Anyone can build that in an afternoon, and many people do, then conclude that agent frameworks are mostly marketing. The difficulty is not the shape. It is every branch the shape hides.

What happens when a tool times out. When the model proposes the same failing call four times. When a tool returns two megabytes. When a user cancels mid-step. When the process restarts with a task in flight. When the accumulated context exceeds the window. When a tool result contains text that looks like an instruction.

That last one deserves emphasis because it is a security property rather than a robustness one. Anything a tool returns is data, never instruction. Web pages, file contents, issue descriptions, and email bodies can all contain text addressed to the agent, written by anyone who can write to those systems. A harness that treats retrieved content as authoritative is exploitable by whoever can put text where the agent will read it, and no amount of prompt wording fixes it.

Seven responsibilities

When I evaluate a harness, mine or anyone else's, I check these seven.

1. Context assembly. What the model sees each step, how it is trimmed, and in what order.
2. Tool execution. With timeouts, argument validation, output size limits, and errors returned as results the agent can act on rather than as crashes.
3. Authority enforcement. Deciding whether an action is permitted, before it runs, by code the agent cannot change.
4. State management. Conversation, task, and progress, durable enough to survive a restart if the work is long.
5. Limits. Steps, wall-clock time, spend, and repeated-failure detection.
6. Recording. What was sent, what was called, what came back, what was decided. Built in, because you cannot record the past.
7. Termination. Knowing when the work is finished, when it has failed, and when it needs a person.

The last one is the most underrated. Agents that cannot recognize failure keep trying, and each attempt costs money and sometimes does damage. Ending cleanly is a feature.

Context assembly decides quality

If I could persuade people of one thing about this layer, it would be this: most problems that look like model problems are context assembly problems.

An agent that picks the wrong table was not shown which one is authoritative. An agent that ignores a constraint had that constraint stated forty thousand tokens earlier, buried in the middle of a long context where attention is weakest. An agent that repeats a failed approach was not shown that it already failed in a form it could recognize.

The diagnostic is cheap. If a stronger model handles the task correctly with the same context, you have a capability problem and a better model or a different tier might help. If every model fails the same way, the context is wrong and no amount of model improvement will fix it.

Three assembly habits carry most of the benefit. Put stable content first and volatile content last, which makes prefix caching possible and costs nothing where it is unavailable. Select rather than fill: ten thousand well-chosen tokens beat a hundred thousand assembled by a similarity threshold. And when sources conflict, say so explicitly and name the authoritative one rather than presenting both and hoping.

Authority is not a prompt

The most consequential mistake in this layer is writing limits into a system prompt and believing they are enforcement.

Do not delete anything. Do not send email without asking. Only touch these tables. These are requests to a system capable of not honoring them. They fail on misinterpretation, they fail on unusual input, and they fail reliably when a document the agent reads contains text designed to override them.

Real authority is enforced outside the model, by the code that executes the action. Capability scoping, so a tool that is not registered cannot be called. Argument constraints, so the tool itself limits what it accepts. Credential scoping, so the credentials cannot perform the forbidden action at all. Approval gates for classes of action, showing the specific action rather than a summary. Process isolation for broad capabilities such as shell access, where argument constraints are meaningless.

The test I apply before deploying anything: if the model were replaced with one that behaved adversarially, what could it actually do? Whatever the answer is, that is the real boundary. Everything else is a preference.

Why I built two of them

Building both Loro and MagAgent looks redundant until you see what each is organized around, and the honest answer is that they solve different problems that I did not want to compromise between.

Loro starts with the controls. Identity, permission decisions over normalized resources, identity-bound approvals with replay protection, sandboxed subprocess profiles, delivered audit records, and governed data access through a catalog. Capability is added inside those constraints. It is aimed at work where being able to demonstrate what happened matters as much as the work happening.

MagAgent starts with memory. A persistent knowledge graph that accumulates across sessions, a broad tool surface including real language servers, and a fast terminal experience. It is aimed at the daily relationship between a developer and their projects, where the value comes from an agent that learns you.

I tried, briefly, to make one thing do both. The result was worse at both. Governance machinery is overhead for a fast daily assistant, and a memory-first design that accumulates freely is exactly wrong when every write needs to be defensible. Those are genuinely different products, and pretending otherwise produced something nobody would choose.

What makes running two acceptable is that the definitions do not live in either. Agents are profiles. Skills are folders. Plans are graphs. Tool connections are protocol servers. The harnesses are interchangeable because nothing important is stored inside them, which is the argument this entire site is making, applied to my own work.

Governed and developer harnesses

I find this distinction more useful than the usual split by domain, because it predicts what a harness will be good at.

Neither is more advanced. Choosing a governed harness for personal coding work produces friction with no corresponding benefit, and choosing a developer harness for work touching regulated data produces a system nobody can defend. The mismatch is the failure, not the category.

Tool design is the real work

An agent can only do what its tools let it do, which makes tool design the highest-leverage work in this layer. It is closer to API design than to prompt writing, and it is where I have changed my mind most.

Specific beats general

A tool called get_open_orders_for_account is chosen correctly far more often than one calledrun_query. I used to prefer general tools on the theory that they were more flexible. They are, and they push the decision onto the model at every single call, where it can be wrong every time. A specific tool encodes the decision once, in code, where it can be tested.

The description is the interface

A tool description is not documentation for humans who will never read it. It is the entire basis on which a model decides whether this tool applies. Say what it does, when to use it, when not to, and what the arguments mean in domain terms. A terse description produces a tool that is used incorrectly regardless of how well it is implemented.

Shape results for reasoning, not for display

Returning a raw API response with sixty fields wastes context and buries the answer. Return the fields that matter, with units, and say what was filtered or truncated. For anything that can return a lot, return a description plus a sample rather than the whole thing, and keep the full result available under a handle.

Errors are input

The model reads errors and acts on them. An error saying an identifier was not found, and what identifiers look like, lets an agent correct itself. A stack trace does not. This means error text is part of the interface and deserves the same care as the success path.

Fewer is better

Tool selection quality degrades as the list grows. Beyond roughly two dozen, models start choosing badly, and the fix is not a stronger model, it is fewer tools in scope for a given task. Scoping tools per task rather than exposing everything is one of the cheapest quality improvements available and one of the least applied.

You will end up with several

This is worth stating plainly because a lot of architecture advice implicitly assumes standardization.

Coding work wants deep repository and symbol awareness. Customer-facing work wants tight latency and conservative defaults. Data work wants query tooling and result handling. Background automation wants durability and retries more than interactivity. These are different products, and a single harness that serves all of them well is rare.

Harnesses also arrive at different times and from different directions. One comes with an editor. One arrives embedded in a vendor product. One is built internally. The realistic goal is not to prevent this. It is to make sure that what matters lives outside all of them.

How I evaluate a harness

• Can I see the actual prompt? Not the template. If not, debugging is guesswork.
• Does it verify, or does it trust the model? A harness that reports success because the final model call said so is not doing the job.
• What are the limits, and are they enforced in code? Steps, time, spend, repeated failures.
• Where does policy live? If the answer involves the system prompt, that is not policy.
• Does it stop well? Watch it fail. A harness that recognizes it is stuck is worth more than one that is slightly better when things go right.
• Would my definitions move? Skills, agent identity, tool connections. If they would not, the harness is a commitment rather than a choice.
• What does it record, and for how long? Because every question I will ask in three months depends on this answer.

Debugging one

Agents fail differently from ordinary software, and the method that works is different too.

There is rarely a stack trace pointing at a line. There is a sequence of individually reasonable steps that added up to the wrong outcome. The only way to find the problem is to read the sequence, which means the trace is the primary debugging artifact and has to contain more than most logging captures by default: the exact context sent at each step rather than a template, every tool call with full arguments, every result including errors, the model and version, and why the loop stopped.

Read it backwards, to the first step where something the agent believed was not true. That divergence point is usually several steps earlier than the visible mistake. Most of the time it turns out a tool gave a misleading result: an empty list where an error was appropriate, a truncated response with no indication of truncation, a column name that meant something other than it appeared to. Fixing the final step does nothing. Fixing the tool fixes the class.

The habit I recommend and do not always keep is reading a sample of real traces weekly, including successful ones. Success traces show wasted steps, unnecessary tool calls, and context bloat that never become visible bugs and entirely determine what the system costs and how fast it feels. Nothing else surfaces that.

What a harness is not

A harness is not a model with extra steps. The model contributes judgment. The harness contributes structure, limits, memory, and accountability, none of which a model can provide about itself.

A harness is not the right home for durable business data. Task state belongs here. Facts belong where they can be queried, governed, and used by systems that are not agents.

A harness is not a substitute for standards. Every harness defines skills, tools, and agent identity somehow. If those definitions live only inside it, the system is portable in principle and captured in practice, which is the outcome all of this work exists to avoid.