Towards the Harness of Embodied Agents

1Introduction

The rise of coding agents [1, 2, 3] has reshaped software engineering. Rather than produce correct code in a single pass, they write, test, read the error, fix, and test again, converging on correctness through a closed loop. The insight is not a better model, but a better harness [4, 5]: the surrounding infrastructure that grounds the model in its environment, mediates its actions through tools, and closes the loop between intent and outcome through evaluation and recovery. Complex behaviors emerge from the interaction between a simple agentic loop [6] and its tools. Plug in new tools and the same architecture that writes code also produces documents, queries external services, and conducts open-ended tasks with the user; capabilities arrive without changing the model or the loop. The harness has become a paradigm [7].

All of this, however, unfolds in digital environments. In the physical world, the building blocks are in place. Embodied foundation models now perform a range of tasks with growing competence, whether navigating cluttered rooms or manipulating everyday objects [8, 9, 10]. Yet a collection of strong atomic capabilities does not constitute a useful robot. What remains open is orchestration: weaving these capabilities into long-horizon behavior in a dynamic world shared with the people it serves. This is exactly the problem coding agents have proved tractable in the digital world. Their answer is not to wait for an omnipotent model, but to build a carefully engineered harness that amplifies what the model can achieve. Does the same paradigm, then, hold in the physical world?

An agent is, in essence, a closed loop of perception and action: it perceives the world, acts on it, and perceives the result. The architecture of coding agents provides a natural blueprint. At each turn the language model selects a tool (e.g. read a file, run a test, edit code), observes the result, and decides the next action. Each robot capability (e.g. navigation, manipulation) can likewise become a callable tool; a language model orchestrates them through the same agentic loop. The design decouples orchestration from execution. The agentic loop accesses policies and platforms only through their tool interfaces, remaining agnostic to their internals. Complex, long-horizon tasks need not be solved monolithically; the agentic loop addresses them step by step, and new behaviors emerge from flexible composition of tools.

At first glance, transferring this blueprint to the physical world should be a simple matter of redefining the tools. It is not. The physical world is far more complex than software. Closing the loop demands two abilities: reading the state of the world, and judging the outcome of an action. Software grants both for free; the physical world grants neither.

What appears “free” to a coding agent is the accumulated product of decades of software engineering. Formal grammars, type systems, and persistent file systems make the state of software readable; exit codes, stack traces, and test frameworks make the outcome of an action verifiable. None of it was built for coding agents; they arrived to find the infrastructure ready, and flourished. Physical environments carry no such inheritance; the missing infrastructure must be built. Nor are the two gaps an arbitrary pair. An agentic loop reaches its environment only through an interface with two directions: actions flow out, and information flows back. The outbound half is the one robotics has spent decades building, which is why policies wrap readily into tools. The inbound half carries exactly two signals: the state of the world, and a verdict on the last action, the same pair the classic agent–environment interface returns [11]. Hence exactly two gaps, one for each missing signal.

We present Thea¹, a harness of embodied agents. Thea inherits the architecture of coding agents: the same agentic loop, capabilities exposed as tools, context, skills, and memory. On this foundation, it supplies the pieces the physical world is missing. Scene Graph as Context restores readability: a persistent, structured model of the scene that the language model can read and reason about as a coding agent reads source code. Evaluation as Exit Codes restores verifiability: it detects when an action should terminate, judges whether it succeeded, and on failure diagnoses the cause, closing the loop that the physical world otherwise leaves open.

Through Thea, we demonstrate that with carefully adapted designs, the harness behind coding agents fits embodied agents well. It wires diverse foundation models, policies, and embodiments into a working whole. The resulting system exhibits the following properties, as shown in Figure 1:

• Extensibility. Capability is organized around tools. Adding a tool extends the agent, a new policy enters as one more tool, and complex behaviors emerge from their free composition rather than being hand-crafted.
• Portability. The model and the body are plug-and-play. A different model or a different embodiment requires no dedicated redesign, because nothing in the architecture is specific to either.
• A bridge between user and world. To the user, the agent is a natural interface. Instructions, questions, and explanations all flow through dialogue. To the world, it runs the classic perception–action loop autonomously, turn after turn. The agent bridges the two, translating the user's intention into fluent execution in the physical world.