What an inference pool asks of the site network

An inference pool is only as good as the path between the robot and the rack. The models can be right and the node can be fast, but if the site network delivers the request late, the robot has already fallen back and the answer is discarded. So before anything else, a pool asks a set of plain things of the network it will live on. This article lists them.

A short, predictable path

The first thing the pool asks is that the path from any robot to the rack be short and, more importantly, predictable. A robot's latency budget is fixed by its vendor, and the pool spends that budget on running the model. Whatever the wire takes comes off the top. A path that is usually quick and occasionally slow is worse than one that is steadily moderate, because the robot cannot plan around the occasional case.

In practice this means the pool is placed in the building's own rack rather than at a remote site, that robots reach it over the facility's network rather than across a public one, and that the hops between the floor and the rack are as few as the building allows. The path never crosses the building's perimeter. A Czar Grid on a large site places nodes so that the farthest robot is still close.

Capacity at the busiest hour

The second thing the pool asks is capacity sized to the busiest hour. Robots meet the unfamiliar in clusters: a trailer arrives, a shift changes, a line turns over. In that hour many robots send perception requests at once, and those requests carry more than a status message. The network between the floor and the rack has to carry that burst without the wire becoming the slow part.

The pool is sized to that hour, as described in Sizing an inference pool for a facility. The network must be sized to it as well. A pool with room to spare behind a congested link is a pool whose robots fall back anyway.

Coverage where robots work

The third thing is coverage. Robots move, and most reach the network wirelessly. Wherever a robot works and cannot reach the rack, it is on its on-board model: safe, and less capable. Dock doors, mezzanines, the far end of a yard, the inside of a trailer, the aisle behind the racking that was added later; these are where coverage thins and where a robot's hesitation is most often a network fact rather than a model fact.

The record on the pool is the map. Every missed budget is recorded with the robot and the time, and misses that cluster by location are coverage gaps. Many sites find that the pool's record is the first honest survey of their wireless coverage they have had.

A boundary the site controls

The fourth thing is a boundary. The pool handles camera feeds, captures, decisions, and the record of all of them, and none of it leaves the building. The network has to make that true. Sites commonly place the pool and the robots on a segment the site controls, so that the path from floor to rack is not shared with traffic that has other destinations.

The pool itself helps. On an air-gapped site the pool has no path out, and updates arrive as signed bundles carried in by media. On a connected site, Local Bridge in Substrate allows only the routes the site declared, to the systems the site already secures, with everything else denied by default. A connection is a decision the site makes, not a default the pool assumes.

What the pool does not ask for

The pool does not ask for a path to the internet. It does not ask for a path to Element 31; customer appliances never connect to the portal, and remote sessions and updates are always initiated by the customer. It does not ask for a path to a robot vendor's cloud, though a site that already secures the vendor's fleet software may declare that route. And it does not ask that the robots change how they reach the network; they use the radio and the path they already have.

What it asks is that the site treat the path from floor to rack as part of the robot, because for the moments that matter, it is. For what that path protects, see The latency budget for a robot served from the rack.