RESOURCES
What a facility inference pool is, how it is sized, and how it serves a floor.
Articles in Inference Pools
- Inference Pools
What an inference pool asks of the site network
A pool is only as good as the path from the robot to the rack. It asks for a short, predictable path, capacity at the busiest hour, coverage wherever robots work, and a boundary the site controls.
· 4 min read
- Inference Pools
A pool that serves people as well as robots
The pool was installed for the robots. It holds the record, the site's documents, and a language model to read them, which is exactly what the people who run the floor have been missing.
· 4 min read
- Inference Pools
The record a pool keeps of every decision
A robot did something and someone wants to know why. The pool writes down what each robot was shown, what answered, what came back, and when, and keeps it on the site's own hardware.
· 4 min read
- Inference Pools
Adding capacity to a Grid without stopping the floor
Capacity is added in the rack, not on the robots. A new Czar is installed, attests what it is running, and is admitted to the pool while the existing nodes keep serving every robot on the floor.
· 4 min read
- Inference Pools
Language models on the facility network
Most of what a facility knows is written for people. A language model on the pool lets robots act on it and lets people ask the floor questions, with no text leaving the building.
· 4 min read
- Inference Pools
Decision models for the moment the robot gives up
A robot's planner stops when it has no path it trusts, and a person walks over. A decision model on the pool takes that moment, chooses within the task's approved bounds, and records why.
· 4 min read
- Inference Pools
Serving perception models from a shared pool
Perception is the model class robots outgrow first. A shared pool in the rack runs the perception models a robot cannot carry, for every vendor's machines, and adapts them on the site's own captures.
· 4 min read
- Inference Pools
How a pool falls back to the robot
When the pool is late or absent, the robot acts on the model it carries, as its vendor shipped it. Safety never left the machine, so a pool that fails leaves a robot that stops, not one that guesses.
· 4 min read
- Inference Pools
When one Czar is enough, and when a Grid is
One Czar is a pool for a small floor. A Czar Grid is one pool for a facility. The choice is a reading of the site: its size, its busiest hour, its tightest budget, and whether an update window is acceptable.
· 4 min read
- Inference Pools
Sizing an inference pool for a facility
A pool is sized to the floor's hardest hour, not to a count of robots. The work, the models, the concurrency, and the tightest latency budget decide whether one Czar is enough or a Grid is needed.
· 4 min read
- Inference Pools
The latency budget for a robot served from the rack
Every robot has a window between needing an answer and having to act without one. Serving from the rack means fitting inside that window every time, and behaving correctly when it cannot be met.
· 4 min read
- Inference Pools
Why robots outgrow their on-board compute
A robot's compute is fixed the day it ships. The models a site wants it to run keep growing after that. The gap is physical, and a pool in the rack is how a floor closes it without replacing the fleet.
· 4 min read
- Inference Pools
An inference pool versus a GPU on every robot
Putting a large accelerator on every robot pays for peak capacity that sits idle most of the shift. A shared pool in the rack sizes compute to the floor, not to the machine.
· 4 min read
- Inference Pools
What is a facility inference pool?
A facility inference pool is shared compute in the building's own rack that serves perception, decision, and language models to every robot on the floor, from any vendor, over the site's network.
· 4 min read