Section 57.1: Learning after deployment

"Deployment is where training becomes a subscription service with consequences."

A Fielded Policy Reading Its Logs

Theory

A single silent weight update that lifts grasp success by three points can, in the same gradient step, push a collision-avoidance margin below the threshold that keeps a forklift-sized robot from hitting a person, and nothing in the loss curve will warn you. That coupling is why post-deployment learning must be governed like a release pipeline rather than treated as spontaneous self-improvement, and why it should be modeled as a staged pipeline:

$$D_t \rightarrow U_t \rightarrow \theta_{t+1} \rightarrow V_t \rightarrow R_t,$$

where collected field data \(D_t\) feeds an update rule \(U_t\), producing candidate parameters \(\theta_{t+1}\), which are evaluated by validation suite \(V_t\) before rollout decision \(R_t\). The key idea is that deployment data and deployment decisions are connected, but not collapsed into one uncontrolled online loop. Figure 57.1B traces these five stages left to right, including the red rollback path that returns control to monitoring whenever a check fails.

Worked Example

Suppose a shelf-picking robot sees more reflective packaging after a supplier change. The new data may justify a candidate perception update, but only after the system verifies that prior carton and bottle skills still work.

def validate_update_pipeline(payload: dict[str, object]) -> dict[str, object]:
    assert payload, "payload must not be empty"
    return payload

update_pipeline = {
    "field_signal": "drop in grasp success on reflective cartons",
    "candidate_update": "fine-tune perception head on corrected examples",
    "validation_panel": ["old carton tasks", "new reflective carton tasks", "safety replay set"],
    "rollout_mode": "shadow_then_canary",
}
print(validate_update_pipeline(update_pipeline))

{'field_signal': 'drop in grasp success on reflective cartons', 'candidate_update': 'fine-tune perception head on corrected examples', 'validation_panel': ['old carton tasks', 'new reflective carton tasks', 'safety replay set'], 'rollout_mode': 'shadow_then_canary'}

The expected output should make the release logic explicit. If the update pipeline does not name retained-task checks and rollout mode, then "learning after deployment" is really just ungoverned retraining.

A policy that improves in the field without a governed pipeline is not learning: it is drifting with good intentions.

Naming those four artifacts is the concrete skill this section aims to build: given any post-deployment adaptation scenario, you should now be able to specify what triggers an update, what the candidate change is, what evidence must pass before rollout, and how rollout is staged, which is exactly what the exercise and lab below ask you to practice.

Shadow mode matters for physical robots for a specific reason. A weight change that looks safe in simulation can still destabilize closed-loop dynamics on hardware. Actuator-response latency, sensor quantization noise, and mechanical wear all create edge cases that offline evaluation cannot replicate. Running the candidate policy in parallel, with its outputs logged but not executed, lets the team observe real sensor streams and real object poses without accepting any physical risk from the new weights.

Mechanically, shadow mode runs the candidate model on every live sensor frame alongside the production model. The production model's action is sent to the actuators; the candidate model's action is recorded to a comparison log. After the shadow window closes, engineers diff the two action streams on the scenario panel, checking for divergences above a threshold before any canary promotion begins.

The update record travels with the model

Logging two action streams is only useful if you can later reconstruct exactly which candidate produced them, which is why the comparison artifact has to travel with the rest of the update record. In production systems, the update object is usually larger than a checkpoint. It should include the exact slice of field data, labeling protocol, adapter or fine-tuning configuration, validation manifest, and the deployment ticket that authorized the shadow or canary run. Tools such as PyTorch training jobs, Weights and Biases or TensorBoard traces, and ROS 2 replay logs are useful here only when they preserve this bundle as one inspectable release artifact rather than scattering evidence across unrelated dashboards.

A strong post-deployment recipe also distinguishes perception adaptation from control adaptation. Suppose a warehouse robot fails because carton appearance changed. The first candidate may then be a narrow vision update with frozen planner and controller interfaces. A LoRA adapter on the vision head touches roughly 0.5% of total parameters, so the 99.5% of weights that govern planning and force control cannot move at all, and a fleet-wide risk becomes a strictly bounded one. If the failure instead comes from timing drift or changed vehicle dynamics, the update path may involve different evaluation panels, different rollback rules, and stricter closed-loop replay before any canary release.

Before reading on, consider: if you could only keep one of the five pipeline stages when shipping under deadline pressure, which would you drop, and what would you lose?

Learning after deployment also reshapes organizational interfaces. Operators, data curators, and release owners need a shared artifact vocabulary so that any update can be challenged and reversed without ambiguity.

Beyond clarifying who can challenge an update, the staged structure also determines whether the update is even tested against the conditions that matter. The staged pipeline matters most when distribution shift is correlated with time of day, seasonal change, or operational context rather than being a random fluctuation. A robot that sees fewer training examples of wet floors in summer will encounter them in winter; if its perception update was triggered only by recent data, the summer model may degrade exactly when wet-floor robustness is most needed. Consider scale: a validation panel covering only the last 7 days of logs may include zero wet-floor encounters, while a panel spanning 90 days typically captures 15 to 40 such cases, enough to detect a regression before promotion. Staging creates a forcing function: the validation panel must be drawn from a distribution that spans known temporal and contextual variation, not just the most recent week. In other words, a panel built only from this week's logs can pass an update that is actually blind to a whole season of conditions; when that panel is narrow, this is called the temporally biased update trap, and the failure will appear as a surprise regression months after the update was promoted.