Section 21.1: Why learning from demonstration matters for robots

"Trial and error is one way to discover how to open a jar. Watching someone do it first is faster, cheaper, and less sticky."

A Robot Watching Very Carefully

Picture a robot arm that has tried for three days to slot a USB connector into a port, firing through tens of thousands of near-misses while a sparse reward stubbornly stays at zero; now picture a human teleoperator solving the same task fifty times in under an hour, each attempt quietly carrying the sub-millimeter alignment secret no reward function ever spells out. That contrast is the whole argument for learning from demonstration, and this section turns it into a usable mental model: first we define the object of study, then we connect it to the agent loop, then we test it with a compact implementation. Figure 21.1A frames the core trade-off, placing learning from demonstration against reinforcement learning on data and sample-efficiency axes to show why demonstrations collapse the exploration problem for contact-rich tasks.

The key question is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

The practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

Worked Example

That inspectable interface contract stays abstract until a concrete task forces every field to take a value, so consider a specific case: a robot arm must insert a USB connector into a port. Reward-shaping this with RL requires defining a numerical reward that fires only on successful insertion, but the connector makes contact at sub-millimeter tolerances and the reward stays zero through thousands of near-misses. A human operator demonstrates 50 episodes via teleoperation. Behavior cloning then treats the problem as supervised regression: for each timestep, it fits a neural network policy \(\pi_\theta(a_t \mid o_t)\) to minimize prediction error against the recorded actions. With 50 demonstrations at 30 Hz, that is roughly 90,000 labeled state-action pairs. Training typically converges in minutes on a single GPU for a policy this small, and in this illustrative run the policy achieves 70% insertion success in evaluation. The gap to 100% is not random; it is covariate shift: the policy visits wrist angles the demonstrator never produced, has no corrective experience there, and drifts into failure states.

from pathlib import Path

dataset_root = Path("robot_demos")
for episode in sorted(dataset_root.glob("episode_*")):
    print("inspect", episode.name)
print("next step: convert demonstrations to the LeRobotDataset format")

Expected output: the printed trace for Why learning from demonstration matters for robots should expose the method configuration, the measured evidence field, and the failure label. If one of those fields is missing or unchanged under the perturbation, the example is not yet an evaluation artifact.

# Inspect the minimum metadata needed before training from demonstrations.
# The check separates robot provenance from policy architecture.
required = {"robot", "camera_hz", "action_space", "operator", "split", "license"}
episode = {
    "robot": "dual_arm_tabletop",
    "camera_hz": 30,
    "action_space": "joint_delta_14d",
    "operator": "teleop_human_A",
    "split": "heldout_task",
}
missing = sorted(required - episode.keys())
print("missing fields:", missing)
print("ready for training:", len(missing) == 0)

Practical Recipe

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

Tying The Method To A Contract

Learning from demonstration earns its keep only under a closed-loop contract that names the observation stream, the state estimate, the action representation, the timing budget, and the evaluation artifact. Without it, a model looks capable in a notebook yet fails the first time a sensor drops a frame or a controller saturates.

Separate the conceptual claim, the systems claim, and the evidence claim. A plausible mechanism, a clean interface, and a closed-loop result are different claims; the section should keep their evidence separate.

Start with a small baseline that logs inputs, outputs, units, timestamps, and termination conditions before moving to Gymnasium or PettingZoo. The library run should keep the same artifact schema, so the comparison remains a same-task evaluation.

1. Write a one-paragraph task contract with observation, action, success, and failure fields.
2. Start with the smallest simulator, dataset, or wrapper that exposes the task contract faithfully.
3. Run one deterministic smoke test and one perturbation test before scaling.
4. Save a single result artifact containing configuration, seed, metrics, videos or traces, and failure labels.
5. Compare methods only when one script evaluates them on the same task panel.

When Why learning from demonstration matters for robots fails, avoid labeling the whole method as weak. First assign the failure to perception, state estimation, planning, control, timing, data coverage, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.