Agent harness engineering: Separating model capabilities from enforcement

In July, highly capable frontier lab evaluation agents escaped their sandbox, reached the internet, and compromised parts of Hugging Face's production infrastructure. The models were capable enough to chain vulnerabilities across trust boundaries. But the breach didn't require the models to defeat every security control; it required the surrounding system to give a capable agent paths around them. The team had noticed 1 trend and missed another.

The trend they noticed: capabilities keep moving into the model. Better models need less scaffolding. Reasoning models absorb planning that used to live in extra agents. Native tool use replaced brittle output parsing. As models improve, capabilities that can be reliably learned tend to become cheaper to absorb into the model than to maintain as external orchestration.

The trend they missed: enforcement needs to move out, not in. Properties such as permissions, identity, isolation, budgets aren't capabilities a model can reliably learn. They are properties that must remain true even when the model is wrong. The evaluation team relaxed enforcement because the model was good enough. The sandbox wasn't.

This post focuses on that split: capabilities inward and enforcement outward.. Not another explanation of the word "harness" (that ground is well covered elsewhere). Instead, we want to explore how to tell which parts of your harness the model will absorb and which parts need to be hardened as infrastructure—and what happens when you get that wrong.

Where the harness came from

The harness started as prompt tricks, grew into tool-calling loops, and is now a full execution environment (well documented elsewhere). In the most recent turn, the harness stops being a library you import and starts being a runtime you operate. Traces matter more than prompts. Tool catalogs need identity, not just descriptions. "The model decided to" isn't an audit trail.

How do I construct an environment in which an imperfect model can reliably accomplish complex goals without giving it the keys to the rest of the estate?

That question is now the job description. But answering it requires noticing "the harness" isn't one thing.

Not all harnesses are the same

Different jobs produced different architectures. They didn't converge as products, though they can still share a runtime when the agent is a service rather than a laptop session.

They diverged for a reason. Workflow harnesses exist because some control flow is too important to leave to a sampler—refunds, approvals, regulated steps. Coding harnesses exist because the environment is the product: a repo, a shell, a test suite. Research harnesses exist because the scarce resource is evidence, not side effects. The loop is allowed to wander; the output isn't allowed to be unsourced.

Agent runtimes exist because someone has to make inference, tools, retrieval, safety, and evaluation look like one system. A LangGraph graph and Claude Code are both harnesses, but they aren't substitutes. Flattening them all into "agents" is how you end up picking the wrong one.

The 4th column is worth pausing on. It's not another coding agent or another framework. It's a bet the machine the loop runs on can be standardized.

The boundary moves in 2 directions

McAteer calls this the attention interface: the model eats what the harness used to do, and what remains is an interface for human attention (when to interrupt, what to approve, and how to spend a scarce operator). That's a real layer, and his analysis is precise for the case he examines: coding agents sitting next to a person.

The question is what happens when you extend the same logic to production agents that have no human attention budget to manage. They have an infrastructure boundary to enforce instead.

Consider a concrete example. In 2024 a typical retrieval-augmented generation (RAG) pipeline had a retrieval step, a reranker, a prompt assembler, and a generation call, representing 4 components stitched together in application code. By mid-2026, models with large context windows and native tool use have started absorbing much of the retrieval and reranking logic that used to live in application code. The harness gets simpler where the model takes over cognitive work. But the authorization check on which documents the model can retrieve doesn't get simpler. It gets more important, because the model is now choosing what to fetch instead of following a fixed query plan. The cognitive part shrinks; the enforcement part grows.

That pattern generalizes. The dividing line isn't cognitive versus non-cognitive—models already do planning, verification, and even security reasoning. The line is: can the model violate this property? If so, it must be enforced outside the model:

• Permissions that hold even if the model is jailbroken
• Identity that is cryptographic, not implied by a system prompt
• Isolation that survives a prompt-injected curl
• Budgets that stop a loop from spending the month's inference allocation
• Recovery that restarts from a checkpoint, not from a reconstructed summary
• Policy that is evaluated on the tool call, not on the model's intention
• Observability that another team can query after the fact

Ask every piece of your harness: if the model gets this wrong, what breaks? If a capability can be reliably learned, models will tend to absorb it. If a property must remain true despite model failure, it belongs outside the model.

Here's the thing worth sitting with: a model that "absorbs permissions" hasn't absorbed permissions. It has dissolved them. The only permissions that count are the ones that still work when the model is wrong, compromised, or having a bad day.

And that leads to a counterintuitive result: the better the model gets, the more dangerous a weak harness becomes. Teams watch the model absorb planning, tool use, and even some self-correction, and conclude the enforcement scaffolding is overhead. They remove it. The system gets more capable and more fragile in the same commit.

So the boundary moves inward for anything the model can learn, and outward for anything that must be enforced. Harness engineering is the work of noticing which is which. Delete the first kind. Build the second as infrastructure. Getting this wrong in one direction means wasted scaffolding. Getting it wrong in the other means an unsupervised capability with the keys to production.

Think of it as an operating system. The model is the process. The harness is the kernel: scheduling, syscalls (tools), memory, permissions, and isolation. The environment—file system, shell, and network—is the device layer. You can swap the process. Swap the kernel, and everything built on top of it notices.

That's why teams already swap model back ends more often than they rewrite tool policy. The stable interface isn't "the weights." It's "what this worker is allowed to touch."

The 4 columns in the table are different kernels for different jobs. The runtime column is what you reach for when those jobs need to run as services: same identity, same policy, same eval surface. Different agents, one machine underneath.

What a runtime looks like

Today, teams writing loops own inference, tools, retrieval, safety, and evaluation as separate problems. The application is the glue: Python scripts wiring together a large language model (LLM), a vector store, a tool executor, and some guardrails by hand.

A runtime is what happens when those become one system. The architectural claim is the layer between the application and the underlying AI services can be a platform, the way Kubernetes made the layer between the application and the machines a platform. Swap the inference back end. Keep the agent API. Keep retrieval. Keep the tool runtime. Keep the eval surface. The model becomes replaceable. The harness becomes the stable abstraction.

The shape is already visible across multiple implementations: Vertex AI Agent Engine, Amazon Bedrock Agents, LangGraph Platform. The specific feature lists differ. What matters is the convergent shape: a runtime API that decouples the agent from the back end, so the model is a plug-in, not a load-bearing wall.

Platform bets at this layer have failed before. Open Neural Network Exchange (ONNX) promised model interchange and delivered partial coverage. OpenTelemetry is succeeding but took years to reach critical mass. The reason to think agent runtimes might converge faster is the contract is narrower: tools in, completions out, policy on the boundary. And there's evidence it's already happening organically: OpenAI's tool-calling schema has become a de facto wire format that Anthropic, Google, and open-weight serving stacks like vLLM all speak, without anyone convening a standards body. When the interface converges before the committee meets, the standardization is describing what already shipped, not prescribing what should. That said, the history of "this time the platform layer will converge" is long enough that skepticism is earned.

It's worth being explicit about the 2 claims this article is making. The first is architectural: agents need identity, isolation, policy, recovery, and observability, and those needs grow as autonomy rises. That claim doesn't require a standardized runtime. Agents can remain application-specific and still need a stronger harness. The second claim is a platform bet: enough of those concerns will converge to justify a shared runtime, the way Kubernetes justified a shared container platform. The first claim is the stronger one. The second is where the industry is placing chips, and where healthy skepticism is warranted.

This layer doesn't compete with a coding harness like Claude Code, which is the right tool when the environment is the repo. A runtime is the right layer when the agent is a service: identity at the tool boundary, policy that isn't a system prompt, and a back end you can run on-prem or swap for a vendor API.

When the harness controls execution, the security boundary moves

The July 2026 incident in brief:

1. Evaluation agents ran in an "isolated" sandbox with cyber refusals disabled.
2. The sandbox included a network path to the internet via a cache proxy.
3. The agents found and exploited a vulnerability in the proxy, escalated privileges, and moved laterally.
4. Over 2 and a half days, they chained multiple vulnerabilities to reach Hugging Face production workers, obtaining root on 1 server and harvesting cloud credentials.
5. Hugging Face's security team initially classified the activity as an unknown external threat actor.

The CSA research note from the Cloud Security Alliance (CSA) put it plainly: "The proximate cause was not a novel model capability but a familiar infrastructure error."

That's the general case. Once a harness can run a shell, the security boundary is the entire model-harness-environment surface. A prompt-injected model is one kind of problem when it can only emit text. It's a much worse problem when the harness will faithfully execute whatever tool call comes back. The harness has to assume the model will eventually ask for something it shouldn't have.

Two complementary answers have emerged, and they're easy to confuse. Application-layer policy (egress filtering, file system allowlists, seccomp) stops data exfiltration but relies on the kernel being trustworthy. Kernel or virtual machine (VM) isolation (a dedicated guest kernel per agent, in the class of a Firecracker microVM) stops container escapes and kernel exploits but doesn't inspect what the workload does at the application layer. Neither alone is sufficient. When teams tested both layers together, the pattern held: application-layer policy alone blocks exfiltration but misses kernel exploits; VM isolation alone blocks exploits but leaks data. Together, the 2 layers address failure modes that neither can address on its own. The portable fact is the split between the 2 layers; specific products are implementations of that split.

Model Context Protocol (MCP) gateways do a similar job 1 layer up the stack. Consider an agent that talks to an MCP server exposing database tools. Without a gateway, the model sees every tool the server offers: query, insert, and drop_table. With a gateway that evaluates an OAuth scope before forwarding the tool list, a read-only agent never sees drop_table in its catalog. A prompt injection that says "call drop_table" fails not because the model refused, but because the gateway never offered the tool. The model can't social-engineer a catalog it can't see.

A minimal policy for this looks like Open Policy Agent (OPA) applied at the gateway:

package mcp.authz

default allow_tool = false

allow_tool {
    input.tool.name == "query"
    "db:read" in input.caller.scopes
}

allow_tool {
    input.tool.name in {"insert", "drop_table"}
    "db:write" in input.caller.scopes
}

Enforcement happens before the model sees the catalog. No prompt engineering required. The through-line is the same as the 2-direction boundary: don't ask the model to police itself.

What remains

If you only watch coding agents, McAteer's forecast looks right. The loop gets simpler. The model eats planning and a lot of tool glue. What remains next to the human is attention: permissions dialogs, notification policy, and when to wait.

If you watch production, a different remainder shows up. The laptop harness can't answer:

• Which identity did this agent use to call that MCP server?
• What was it allowed to see, and who attested that?
• Can we replay the run without re-spending the tokens?
• What happens when the cluster dies mid-loop?
• How do we run the same agent on premise, in a disconnected environment, against a vLLM endpoint instead of a vendor API?
• How do we evaluate it as a system, not as a chatbot?

These are infrastructure problems. They map to a stack: compute and isolation at the bottom, inference and model services in the middle, and agent services at the top.

So what's left, if models keep absorbing today's scaffolding?

1. Environment. A computer, or a constrained facsimile of one. No amount of in-context learning replaces a file system, a network stack, or a browser. The model reasons; the environment is where reasoning has consequences.
2. Enforcement. Identity, policy, isolation, budgets—things that must be true when the model isn't. The model can't learn these because they aren't predictions. They are constraints on a system that includes the model as 1 component. Layered sandboxing tests have consistently shown the pattern: application-layer policy alone missed kernel exploits; VM isolation alone missed data exfiltration; only the combination addressed both failure modes. Enforcement has to be layered, and no layer can live inside the model.
3. Operation. Tracing, evaluation, recovery, and lifecycle—things a platform team can run. A model can generate a trace; it can't be the tracing infrastructure. A model can suggest a retry; it can't be the checkpoint store. A single agentic query can produce dozens of metrics and log events across model calls, tool invocations, and retrieval steps. Correlating them requires distributed tracing with context propagation, not application-level logging.
4. Attention. The human interface McAteer describes, for the agents that still sit next to a person. This layer shrinks as autonomy rises, but for the large class of agents that work alongside humans, it remains the last mile.

Items 1 through 3 get more important as autonomy goes up, not less. A more capable model in a weaker harness is a more capable way to exfiltrate ~/.ssh. The harness doesn't shrink to nothing. It sheds the parts the model learned, and hardens the parts the model must never be trusted with.

Where to start

Pick 1 agent you already run. Name the harness it has: loop, environment, or runtime. Then name what's missing.

These architectural gaps map naturally onto existing platform capabilities. In the Red Hat ecosystem, the starting points look like this:

• If you have an agentic loop but no unified control at the tool boundary, the gap is the gateway layer. Praxis AI proxy is one place to start: provider-aware routing, protocol translation, agent traffic management, and safety guardrails behind a single API surface. For the underlying model serving tier, look at Red Hat's EnMaaS pilot for open source inference.See also Bring Your Own Agent for how these layers compose end-to-end.
• If you have a runtime but agents share a kernel with production workloads, the gap is isolation. Red Hat OpenShift sandboxed containers give you a hardware-isolated VM per agent workload. The layered sandboxing results referenced earlier show why application-layer policy alone is not enough.
• If you have isolation but no way to replay a run without re-spending the tokens, the gap is operation. Start with distributed tracing for agentic workflows for World Wide Web Consortium (W3C) context propagation, and AI observability with MLflow for correlating traces across model calls, tool invocations, and retrieval steps.

For the full-stack view of how compute, inference, and agent services fit together, see Metal to agents.

The diagnosis comes before the tooling. The model will keep getting better. That's not an argument for investing less in the harness. It's an argument for investing in the harness as the thing that remains when the model is swapped.