Reviewing an incident from a local record
When something goes wrong on the floor, a Czar Grid gives the site a complete local record to review: the task version, the decisions served, the fallback, the robot's own stop. No vendor needs to be asked.
· 5 min read
IN SHORT
An incident on a Czar Grid floor is reviewed from the site's own local record. The team fixes the timeline, replays the task run, reads what the pool served and whether it met budget, checks whether the robot fell back or stopped on its own limits, and confirms node integrity at the time. The findings belong to the site, and no outside party holds a piece of the story.
An incident on a robot floor produces the same first question every time: what happened. On a floor served by a vendor's cloud, part of the answer is somewhere else and arrives when the vendor provides it. On a floor served by a Czar Grid, the answer is in the building. The task that was running, the decisions the pool served, whether they arrived within the robot's budget, whether the robot fell back, and what each node was running at the time are all in one local record the site owns. This article is a method for reading it.
Fix the timeline first
Begin with the window. The safety officer or the shift lead establishes when the incident began and ended in the floor's own terms: the robot slowed here, stopped there, a person was in this zone. The record is then read against that window, not the other way around. Starting from the record and searching for something interesting is how reviews drift.
With the window fixed, the three threads of the record are pulled for it. The task thread shows which version was running on the robot and under which bounds. The serving thread shows each request the robot made to the pool, what was served, and whether it met budget. The integrity thread shows whether the nodes involved were verified clean. Lay them side by side in time.
Separate the layers
A clean review keeps three layers apart and asks what each did.
The task layer. Was the robot running an approved version. Were the bounds in force appropriate for the zone it was in. Did the task's own stops and limits apply where they should have. If a bound was wrong, the finding is a task finding, and it leads to a new version through the normal approval path.
The pool layer. What did the pool serve the robot in the window, and did it arrive in time. If the pool served a perception or decision output that was wrong for the situation, that is a pool finding, and it points at evaluation, at the model, or at adaptation on the site's own captures. If the pool missed budget and the robot fell back, that is a different finding, about load or network path, and the robot's behavior afterwards is its own on-board model's.
The robot layer. Did the robot stop, slow, or limit motion on its own safety as the vendor built it. Safety stays on the robot without exception, so a stop is the vendor's limits working, not the pool intervening. The Grid serves perception and decisions; it does not hold the stop. A pool that fails leaves a robot that stops, not one that guesses, and the record should show which of those occurred.
Replay the run
Agency Studio makes every run replayable. The team replays the run across the incident window and watches the decision chain: what the robot was served, what it did next, where the task bounds engaged, where the robot's own limits engaged. Replay is what turns a timeline into an explanation.
It also keeps the review honest. People remember incidents in terms of outcome; the replay shows the sequence. A robot that stopped short of a person did so for a reason the record can name, and that reason is often not the one the first witness assumed.
Check integrity at the time
Watchman's result for each node involved is read for the incident window specifically, not just for the current moment. A node that was verified clean then and is clean now can be set aside. A node whose result changed around the window moves to the center of the review, and it does not serve again until the site understands why. In most incidents this check takes a moment and confirms the pool was what it claimed to be, which is itself a finding worth writing down.
Write the findings where the record lives
The review produces findings and, usually, actions: a task revision, a budget adjustment, a route check with the integrator, a conversation with a vendor about a fallback, a procedural change for the floor. Each action lands with a named role, and task revisions go through the same approval by the automation lead and the safety officer as any other version.
The findings are recorded beside the incident in the site's own record. Later, when management, an insurer, or an auditor asks what happened and what the site did about it, the answer is a record of the incident, the replay, the findings, the revision, and the approval, in order, written by the people who were there. The site does not have to request anything from anyone to produce it.
Consider what the site did not have to do. It did not have to ask a vendor's cloud for data it does not control. It did not have to wait for a support ticket to surface a log. It did not have to reconcile a remote account of events with what the floor saw. The record is local because the pool is local, and the review is the site's own work from beginning to end. If the site chooses to involve Element 31, it does so by opening a customer-initiated session through the portal, on its own terms and after it has already read its own record. Monitoring a pool that does not phone home covers the habits that make this kind of review possible.
Questions this raises
- Do we need the robot vendor's logs to understand an incident?
- The vendor's own logs cover the machine's internals and its safety behavior, which the vendor owns. The Grid's record covers what the robot was served, under which task version, and whether it fell back. A full review often reads both, but the site is never waiting on a vendor to learn what the pool did.
- Can the record be altered after an incident?
- The record is written as events happen and is the operator's own. Sites that care about this treat the record as they treat other controlled operational records, with access governed by their own policy. Its value in a review comes from having been written at the time, not reconstructed afterwards.
- What if the pool was not involved at all?
- The record will show that too: no request from the robot in the window, or a request answered within budget followed by a stop on the robot's own limits. Ruling the pool out quickly is one of the more useful things the record does, because it lets the review move to the right layer.
KEEP READING