Why robots outgrow their on-board compute

A robot's compute is decided on the day it is designed and fixed on the day it ships. The models it is asked to run are decided later, by the site, and they keep changing for as long as the robot works. That mismatch is the whole story of why robots outgrow their on-board compute.

The chassis is fixed; the models are not

When a vendor designs a robot, it picks an on-board computer that fits the mass budget, the thermal budget, and the battery. It then trains or licenses models that fit that computer. On the day the robot ships, the two match.

The site that buys the robot has different ambitions. It wants the robot to handle a new product line, to read labels it has never seen, to work alongside a second vendor's machines, to understand a work order written for a person. Each of those wants a larger or different model. The robot's computer does not grow to meet them.

The hard moments need the large models

A robot spends most of its time doing what it has done before: the known aisle, the known pick, the known pallet. Small models handle that, and they handle it on board. The moments that stop a robot are the unfamiliar ones, and the models that handle the unfamiliar are the large ones.

That is not a coincidence. Handling the unfamiliar means recognizing more kinds of thing, reasoning over more context, and reading language rather than barcodes. All of that costs parameters and memory. The robot that needs the large model most is the robot standing still in an aisle waiting for a person, and the large model is exactly what its on-board computer cannot hold.

The ceiling is physical

On-board compute cannot simply be scaled up. A larger accelerator weighs more, and the drive system and battery pay for every gram across every shift. It produces more heat, and a sealed enclosure on a moving machine has few ways to shed it. It draws more power, and the robot's working time between charges shrinks to match.

Humanoids feel this first and hardest. A machine that balances on two legs has almost no slack in mass or battery, and every unit of compute it carries is a unit of battery it cannot spend on walking. Mobile robots and pick arms have more room but the same shape of ceiling. No vendor roadmap removes the physics; it only moves the line.

The cloud is too far

The obvious answer is to send the hard question somewhere with more compute. For an office application that works. For a robot in an aisle it does not, because the answer has to arrive before the robot's next decision, and a round trip to a distant cloud does not reliably do that. Beyond timing, the question carries a camera feed of the floor, and many sites will not send that out of the building at all.

So the question becomes where to put compute that is large enough for the model, close enough for the deadline, and inside the site's own boundary.

The pool takes the overflow

An inference pool in the facility's rack is the answer to that question. It takes the large models the robot cannot carry and serves them over the site's network, inside each robot's latency budget. The robot keeps its own model for reflexes and for everything its vendor built into it, and falls back to it whenever the pool cannot answer in time.

At Element 31 that pool is Czar, one sealed node for a small floor or several joined as a Czar Grid for a whole facility. Substrate, the serving system on the Grid, runs inference across the pool and holds each robot's budget. When the site wants a larger model, it updates the rack; the fleet stays as it is. For what that looks like in the serving path, see The latency budget for a robot served from the rack.