What is a facility inference pool?
A facility inference pool is shared compute in the building's own rack that serves perception, decision, and language models to every robot on the floor, from any vendor, over the site's network.
· 4 min read
IN SHORT
A facility inference pool is shared compute installed in a building's own rack that runs the perception, decision, and language models robots cannot carry on board and cannot wait on a distant cloud for. Every robot on the floor, from any vendor, draws on it over the site network, while safety functions stay on each machine.
A facility inference pool is compute that belongs to the building. It sits in the building's own rack, runs the models that robots on the floor need but cannot carry, and answers them over the facility's network. Every robot on the floor, whatever vendor built it, draws on the same pool.
The idea is simple to state and specific in its consequences. The models that make a modern robot useful, the ones that recognize an unfamiliar pallet, choose between two bad paths, or read a work order written for a person, have grown past what a robot's chassis can hold. Sending them to a distant cloud works in a demonstration and fails on a floor where the answer is needed before the robot finishes its next step. A pool in the rack is the third option: large enough for the model, close enough for the deadline.
Where the pool lives
The pool is installed in the facility, not near it. At Element 31 the appliance is Czar, a sealed node that goes into a rack the building already owns. One Czar is a pool for a small floor. Several Czars joined together form a Czar Grid, one pool for a whole facility, still Czar, not a different machine.
Because the pool sits inside the site's own network, the path from a robot to an answer is the path from the floor to the rack and back. It does not cross the building's perimeter. That is what makes the latency workable and what makes the privacy posture simple: the camera feed, the capture, the decision, and the record of that decision all stay on hardware the operator owns.
What the pool serves
The pool serves three kinds of model. Perception models turn what a robot's cameras and sensors see into things the robot can act on: this is a pallet, that is a person, this label is damaged. Decision models take over at the moment the robot's own planner runs out of options and a choice has to be made. Language models let the robot read the site's documents, interpret an instruction written for a person, and explain in plain words what it did and why.
These are the models that are too large to carry and too urgent to send away. A robot's on-board compute is sized for what it does every second. The pool is sized for what the floor needs in its hardest moments, and it is shared, so no single robot has to carry capacity it uses only occasionally.
What stays on the robot
Safety stays on the machine. Emergency stops, speed limits, reflexes, and motion limits remain on the robot as its vendor built them. The pool does not take over those functions and is not in the path of them.
This division is deliberate. The robot keeps its own on-board model for reflexes and uses it as a fallback whenever the pool cannot answer within the robot's latency budget. A pool that fails, or a network that drops, leaves a robot that stops, not one that guesses. The pool improves what the robot can do; it never becomes the thing the robot depends on to be safe.
One pool for many vendors
Most floors are not single-vendor. A warehouse may run mobile robots from one company, pick arms from another, and a trial of humanoids from a third. Each vendor ships its own fleet software and its own on-board model. None of them planned for the others.
A facility pool is the layer under all of them. Each robot connects over the site network, declares what it needs, and is served within its own budget. The serving system that does this at Element 31 is Substrate, which runs inference across the pool, holds memory and retrieval over the site's documents, and keeps the record of every decision it serves. The site owns one pool and one record, rather than a separate compute story for every vendor on the floor.
Why the building owns it
Robots arrive and leave. Vendors change. The building stays. Putting the pool in the rack makes it a property of the site rather than of any one fleet, which is why it can serve the next vendor as readily as the current one, and why the site can answer, from a record it holds, what every robot on its floor was told and why.
The pool can run air-gapped, with no path out and updates carried in as signed bundles, or connected only to the systems the site already secures: the vendors' fleet software, the warehouse system, the controllers. Posture is a choice per site, not a different appliance. For how the pool is matched to a floor, see Sizing an inference pool for a facility.
Questions this raises
- Does a facility inference pool replace the computer on the robot?
- No. The robot keeps its own on-board model for reflexes, stops, and motion limits, exactly as its vendor shipped it. The pool adds the larger models the robot could not carry, and the robot falls back to its own model when the pool cannot answer in time.
- Is the pool a cloud service hosted somewhere else?
- No. The pool is a sealed node, or several, installed in the facility's rack and reached over the facility's network. It can run air-gapped or connected only to systems the site already secures, and nothing it processes leaves the building.
- Do all the robots have to come from one vendor to share a pool?
- No. The pool serves every robot in the building from any vendor. Mixed fleets of mobile robots, pick arms, and humanoids draw on the same pool, each with its own latency budget.
KEEP READING