Towards the Harness of Embodied Agents

7Discussion

This work took an exploratory step towards the harness of embodied agents, realizing the paradigm of coding agents as one working system, Thea. Each component was adapted to the physical world, from the loop and tools to context and memory. The two properties the world does not grant, readability and verifiability, were rebuilt as the scene graph and the evaluator.

We offer Thea as evidence that embodied AI is entering the paradigm shift that software agents have already undergone, where the layer between the model and its environment has come to matter as much as the model itself [5, 7, 15]. In fact, an embodied agent needs such a system, rather than just a capable model, even more than a coding agent does, as what it faces is far more complex. The world changes and surprises on its own schedule, observations are always partial, executions fail and cannot be rolled back, and the body's limits shape what is possible.

We believe this shift will fundamentally change how embodied agents are built:

• Reliability becomes a systems problem. It can be engineered in the harness without touching model weights, and the gain compounds with the model's own progress, since a stronger model raises what the same harness delivers.
• Capability becomes additive. A new policy enters as one more callable tool, and a new body as one more embodiment profile, with nothing retrained around them, so policy builders and system builders can advance in parallel.
• The loop becomes a training target. Reading a scene graph, choosing among tools, admitting failure and retrying are today the work of a general language model. They are learnable behaviors, and natural objectives for the next generation of embodied foundation models. VLAs and WAMs enter the loop as tools today, and they can evolve towards the objectives the loop sets.

Necessary as we have argued each component of the harness to be, we do not claim any of the specific implementations to be final. Indeed, each has its own limitations. The scene graph is limited by the perception beneath it. Positions are coarse, and associating observations of the same object over time still admits errors. The evaluator approximates the exit code it replaces, and its verdicts carry false positives and false negatives that a return value does not. And the loop takes time to decide. Every decision passes through a language model, and the system does not yet act in real time.

Each of these limits marks a direction the framework can grow. For the scene graph, we can explore representations of the world that are more efficient and more durable, tracking entities through change, towards a household memory of where things are and who moved them. The evaluator will always carry an error rate, but the error rate can be trained down, with large collections of judged outcomes, with the habit of gathering more evidence before ruling, and in the long run jointly with the policies it judges. A compact evaluation model, trained for this one job and run locally, could then rule on every action without a round trip to a frontier foundation model. A robot, more than a coding agent, has reason to run its models on board, so a compact model trained for the loop, accelerated at the edge, would bring decisions towards real time. Memory also changes what the agent is to the people it serves. It can grow more personalized and more proactive over time, learning a household's habits and offering help before being asked. Such familiarity raises new questions of privacy, trust, and initiative that deserve as much care as capability. And deployment itself feeds improvement. Interaction with the real world produces exactly what policy learning consumes, ready for the agentic self-improvement engines built by recent work [61, 62], closing a flywheel from experience to policies to stronger single steps. We envision Thea growing with each turn of this flywheel, into part of the infrastructure that embodied agents take for granted.