Serving perception models from a shared pool

Perception is where robots run out of room first. Knowing that a thing is a pallet, a person, a damaged label, or a part that is not in the catalog takes a model that grows with every new thing it must recognize, and a floor is always adding new things. A shared pool exists, more than for any other reason, to serve perception models that no longer fit the robot.

What the robot sees and what it sends

A robot's cameras and sensors produce more than its on-board model can interpret. The on-board model handles what it was trained for: the known aisle, the known bin, the obstacle shape it has seen before. When it meets something outside that, it has a choice between acting on a guess and asking.

With a pool in the rack, it asks. The robot sends what the request needs over the site's own network to the pool, and the pool runs a perception model large enough to answer: what is in the scene, where, and with what confidence. The answer comes back inside the robot's latency budget, and the robot acts on it. The path is from the floor to the rack and back; the feed never leaves the building.

One pool, many kinds of eye

A floor's robots do not share sensors or vendors. Mobile robots carry one arrangement of cameras, pick arms another, humanoids a third. Each vendor trained its on-board model on its own hardware. A pool that served only one of them would be a vendor feature. A pool that serves all of them is a facility feature.

Substrate holds the perception models the floor needs and serves each request against the robot's own budget. A request from a humanoid mid-step is scheduled ahead of a request from a stationary arm. A mixed fleet is the normal case, not a complication, and it is one reason a pool is sized smaller than the sum of its robots' needs: different robots meet the unfamiliar at different moments, so the pool's peak is lower than the sum of theirs.

Perception that learns the site

A general perception model knows what a pallet is. It does not know this site's pallets, this site's packaging, the lighting on this floor at the end of the day, or the particular way a part from this plant's supplier arrives. The gap between general and local is where most of a floor's hesitation lives.

The pool closes that gap by adaptation on the site's own captures. Frames the site's robots have already taken are used to fine-tune the perception model on the pool itself. Nothing is uploaded anywhere. The adapted model is evaluated against the site's own record before it serves, and the evaluation is recorded. The result is a perception model that knows this floor, held on this floor.

What stays on the robot

The robot keeps the perception it needs for reflexes. Obstacle avoidance at the scale of the next footstep, the geometry that keeps an arm out of a person's path, the sensors that trigger a stop: these are on the machine, as the vendor built them, and the pool is not in their path. The pool serves the perception that decides what to do next; the robot keeps the perception that decides whether to move at all.

When the pool cannot answer in time, the robot falls back to its on-board model and acts conservatively. A missed perception answer slows a robot. It does not endanger anyone.

The record of what was seen

Every perception answer the pool serves is written to the record: the robot, the request, the model and its version, the answer, the time. This is what lets a safety officer or an auditor reconstruct what a robot was told about a scene before it acted. The feed, the capture, and the record stay on the operator's hardware inside the operator's network.

A Czar in the rack is a perception pool for a small floor. A Czar Grid is one for a facility. In both cases the robots see one pool, and the site holds one record. For what the pool does when perception is not enough and a decision is needed, see Decision models for the moment the robot gives up.