Sizing an inference pool for a facility

Sizing a pool is not counting robots. A floor of many robots doing known work asks the pool very little; a floor of a few robots handling unfamiliar freight asks it a great deal. The pool has to be sized for what the floor demands at its hardest moment, and that demand is a function of the work, the models, and the budgets, with the robot count a long way down the list.

Start with the work, not the fleet

The first question is what the robots on this floor will be asked to do that their on-board models cannot. A fulfillment floor whose mix changes daily needs perception for things it has never seen. A plant whose humanoids work beside fixed arms and controllers needs decisions about shared space. A terminal needs inspection across a large outdoor area. Each of these is a different demand on the pool.

From that comes the list of models: which perception models, which decision models, whether a language model is needed for reading work orders and site documents. The size and number of models the pool must hold ready sets the floor of the pool's size. A model that is loaded and waiting is a model that can answer inside a budget; one that has to be fetched cannot.

Concurrency at the busiest hour

The second question is how many robots will need the pool at the same time, and when. Hard moments cluster. A shift change, a trailer arriving, a line changeover, a wave of unfamiliar inbound: these are the hours when many robots hit the edge of their own models together. The pool is sized to that hour, not to the quiet one.

Mixed fleets help here. Mobile robots, pick arms, and humanoids reach their hard moments at different points in the same hour, so the combined peak is lower than the sum of each fleet's peak. A single-vendor fleet of identical machines moves more in lockstep, and its peak is sharper.

The tightest budget wins

The third question is the shortest latency budget among the robots asking. A humanoid mid-step cannot wait for a pool that is draining a queue of pick-arm requests. Substrate schedules against each robot's budget, so slack requests absorb delay, but the pool still has to have enough capacity that the tightest budget is met at the busiest hour.

In practice this means sizing for the tight-budget demand first and letting the slack-budget demand fill in around it. A floor with many humanoids is sized differently from a floor with the same number of pick arms, even if the model list is the same.

One Czar, or a Grid

With demand understood, the answer takes one of two shapes. One Czar in the rack is a pool for a small floor: a single-building operation, a trial area, a section of a larger site. Several Czars joined as a Czar Grid are one pool for the whole facility, with capacity to spare for the busiest hour and the tightest budget. The Grid is still Czar, configured for the site, not a second kind of machine.

The site does not have to get this exactly right on the first day. Capacity is added to a Grid in the rack without changing anything on the floor, as described in Adding capacity to a Grid without stopping the floor. The commercial model matches: one subscription for the configured Grid, hardware, platform license, and sustainment in one line, fixed for the term. More robots on the pool do not move it.

Leaving room for what comes next

The pool outlives the fleet. Vendors will change, robots will be added, and models will get larger. Sizing should leave headroom for the next vendor's robots and the next generation of model, because a pool that is full on the day it is installed will be short the day the floor changes.

E31 Consulting does this sizing with the operator's automation team, the integrator, and the robot vendors on site, since each holds part of the demand picture. Sizing a pool is a conversation about the floor's hardest hour, and the people who run the floor know when that is. For the decision between a single node and a Grid, see When one Czar is enough, and when a Grid is.