The latency budget for a robot served from the rack

Every robot has a latency budget, whether or not anyone has written it down. It is the time between the moment the robot needs an answer and the moment it must act with or without one. Serving a robot from the rack is an exercise in fitting inside that budget, every time, and in behaving correctly when the budget cannot be met.

Where the budget comes from

The budget is a property of the robot, not of the pool. A mobile robot driving an aisle at its working speed can wait a certain time for a perception answer before its own planner must commit to a path. A humanoid taking a step has a shorter window. A pick arm pausing above a bin has a longer one. The vendor set these limits when it built the motion profile and the safety case, and they do not change because a pool arrived.

The pool's job is to respect those limits, not to negotiate them. The site can tell the pool to be stricter for a given task. It cannot tell the pool to be looser than the robot's own tolerance, because the robot will have acted already.

Why the rack is close enough

The budget rules out a distant cloud for perception and decisions. A round trip across the public network has variable timing the robot cannot plan around, and the floor is not a place for variable timing. The rack in the same building is different in kind. The path is from the robot's radio to the site network to the rack and back. It never crosses the building's perimeter, and it shares a path only with the site's own traffic.

That is the physical reason a facility inference pool works where a remote one does not. Czar sits in the building's own rack. A Czar Grid joins several Czars into one pool for a larger facility, still on the same network, still inside the same walls.

Holding a budget per robot

A floor is not one robot with one budget. It is many robots, from several vendors, each with its own budgets for its own kinds of request. The serving system on the Grid, Substrate, holds each robot's budget separately. A perception request from a humanoid mid-step is scheduled ahead of a language request from a pick arm that is waiting anyway. Both are served, but the pool knows which one cannot wait.

This is why a pool can be shared without the sharing showing up on the floor. Capacity is allocated against deadlines, not first come first served. When demand is high, the requests with slack absorb the delay, and the requests without slack are still answered on time.

When the budget cannot be met

Sometimes the pool cannot answer in time: the network is congested, the node is under unusual load, a link has dropped. The design rule is that a late answer is not an answer. The robot does not wait past its budget. It acts on the model it carries, which for a vendor-built robot means slowing, stopping, or taking the conservative path its vendor defined.

The pool, for its part, does not deliver the late answer into a situation that has moved on. It discards it and records that the budget was missed, with the robot, the request, and the time. The site can read that record and see where the network is tight or where demand is outgrowing the pool. Nothing about a missed budget makes the robot less safe; it only makes the robot briefly less capable, which is the correct direction for the failure to point.

Designing the floor around the budget

Once the budget is understood as the contract between the robot and the pool, several site decisions follow. The site network from floor to rack is sized so that the pool's own compute, not the wire, is the part of the budget that varies. Robots that need the tightest budgets are placed on the parts of the network with the fewest hops. And the record of missed budgets becomes the first place the automation team looks when a robot on the floor seems hesitant.

For how fallback works when the budget is missed, see How a pool falls back to the robot. For what the budget asks of the wire itself, see What an inference pool asks of the site network.