Towards the Harness of Embodied Agents

ATool Registry

Table 4 lists the Astribot S1 tool registry. Its entries follow the protocol of §3.3: physical capabilities use deployment backends, while lightweight context and interaction tools run in-process. Slash-separated lists are the admissible values; quoted strings are free-form descriptions; refs like desk_24 name Scene Graph entities.

Table 4. Tools deployed on the Astribot S1, grouped by function. evaluate_run is registered alongside them but invoked by the harness through a post-hook, never selected by the model (§4.2).
ToolDescriptionBackendParameterValue
Navigation
navigate_to Approach an object or location. SysNav [28] target e.g. “drink area”/desk_24
move_base Translate or rotate within safety bounds. Astribot S1 SDK direction forward/backward/left/right/turn_left/turn_right
distance_m 0.03–0.80 m
angle_deg 3–30°
Manipulation
pick_up Grasp a localized object. CaP-X/ACT/π0.5
[32, 39, 10]
obj e.g. “orange juice”/bottle_2
place Place the held object on a table or in a box. CaP-X/ACT/π0.5 target e.g. “table”/table_2
open_drawer Open a drawer. CaP-X/ACT/π0.5 container_ref e.g. cabinet_87
drawer_level higher/lower
close_drawer Close a drawer. CaP-X/ACT/π0.5 container_ref e.g. cabinet_87
drawer_level higher/lower
trash_drop Drop the held object into a bin. CaP-X/ACT/π0.5 trash_ref e.g. trash_can_54
Perception
tilt_head Tilt the head camera to a given pitch angle. Astribot S1 SDK pitch_deg 0–60°
get_object_relations Query an object's spatial and semantic relations. Scene Graph obj e.g. cup_2
relation (optional) on/inside/holding/near
get_image Retrieve stored images of an object. Scene Graph obj e.g. cup_2
Skills
load_skill Load the instructions of a registered skill. Thea name e.g. tidy-workspace
User Interaction
query_user Request and await clarification. Lark question e.g. “No plain water; which drink should I bring?”
candidate_refs (optional) e.g. bottle_3
observation_views (optional) e.g. torso_rgbd
notify_user Send a non-blocking update. Lark message e.g. “I found an empty orange-juice bottle on desk…”
notification_type (optional) progress/warning/completion
[28]: Zhu et al. (2026), SysNav: Multi-Level Systematic Cooperation Enables Real-World, Cross-Embodiment Object Navigation. [32]: Fu et al. (2026), CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. [39]: Zhao et al. (2023), Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware. [10]: Black et al. (2025), π0.5: A Vision-Language-Action Model with Open-World Generalization.