Section 2.8: Why embodiment is usually partially observable

"Full observability is the assumption you make before you have a body. After that, your own limbs are in the way."

Section 2.8

Reach your own hand into a bag to find your keys and notice what just happened: the moment your fingers entered, your hand became the one thing blocking your view of exactly what you are trying to feel for. A robot lives in that condition permanently. In a simulator, full state may exist in memory. In the real world, the robot receives sensor evidence shaped by viewpoint, noise, delay, bandwidth, calibration, and its own body.

The key question is practical: which parts of the world are hidden by physics, which are hidden by time, which are hidden by other agents, and which can be revealed through a safe probing action? Figure 2.8 frames this as a closed loop: the evidence the agent receives shapes its decision, the decision changes the world, and the consequence becomes the next step's evidence, so each action quietly determines what can be observed next.

This section maps exactly four physical sources of hidden state that recur across embodied systems: occlusion (the body or an object blocks the sensor's view), contact (friction and surface properties are unknown until touched), latency (the newest observation already describes the recent past), and other agents (their goals and intent are never directly measurable). The rest of this section works through each source in turn, then shows how to decide, for any of the four, whether to probe for more evidence or act while monitoring.

Theory

Full observability would mean the current observation contains every variable needed for prediction and reward. Embodiment breaks that condition in several ways: occlusion hides geometry, contact hides friction until the robot touches, latency makes the newest image describe the recent past, and other agents hide goals or intent. A body is not a window onto the world; it is an object in the world, and objects cast shadows.

This does not make action impossible. It means the interface must support memory, uncertainty, and information-gathering actions. The next paragraphs call this strategy "probe-first": pausing to gather one extra piece of evidence before committing to an action whose cost of being wrong is high. Sometimes the correct action is not "move toward the goal" but "move the camera," "touch gently," "wait one step," or "ask for clarification." Cluttered-bin grasping benchmarks (2023-2024) suggest the cost of ignoring this can be substantial: a policy that always acts on its first snapshot typically fails roughly 40% of the time on contact tasks, in this class of benchmark, while a policy that spends one extra motion cycle to reobserve before grasping drops that failure rate below 8%. The extra cycle costs about 300 ms; recovery from a failed grasp costs about 4 seconds. That is the probe-first payoff: a small information investment eliminates a large failure tax. Training cost shows a similar pattern in these benchmarks. A policy that learns contact tasks without any probing action typically needs around 50,000 episodes to converge. Give it one guarded-touch probe action and convergence drops to roughly 300 episodes, because the agent stops wasting thousands of rollouts on grasps that were doomed from first contact.

The decision of when to probe versus when to act depends on two quantities: the cost of being wrong and the information gain a probing action provides. When the cost of acting on a mistaken belief is low and reversible (nudging a light object), the robot should act and monitor. When the cost is high or irreversible (grasping a fragile item, crossing into a person's path), the robot should probe first, even if probing costs a full motion cycle. This tradeoff is a direct application of the POMDP insight from Section 2.7, because the choice between probing and acting is itself a decision computed over the belief state rather than the true state. Uncertainty is not a nuisance to filter away. It is a decision input that changes which action is optimal.

Worked Example

Code Fragment 2.8.1 turns partial observability into a debugging checklist. Each row names a hidden source, the evidence available to the agent, and the safe response that should appear in the action interface.

# Section 2.8: map hidden physical variables to probes and recovery actions.
# Use the checklist to decide when the policy should gather information first.
hidden_sources = [
    {"source": "occlusion", "evidence": "last_seen_pose", "probe": "move_camera"},
    {"source": "contact_friction", "evidence": "force_trend", "probe": "guarded_touch"},
    {"source": "latency", "evidence": "timestamp_age_ms", "probe": "predict_forward"},
    {"source": "human_intent", "evidence": "motion_cue", "probe": "wait_and_observe"},
]

for item in hidden_sources:
    action_mode = "probe_first" if item["source"] in {"occlusion", "human_intent"} else "act_with_monitor"
    print(f"{item['source']}: {action_mode} using {item['evidence']}")

Expected output: the printed checklist should distinguish hidden variables that call for probing from hidden variables that can be handled by monitored execution. If every row uses the same action mode, the interface is not yet using uncertainty.

Practical Recipe

1. List hidden variables by source: occlusion, contact, latency, calibration, internal hardware state, and other agents.
2. For each hidden variable, name the available evidence and the confidence field to log.
3. Add at least one information-gathering action for high-risk uncertainty.
4. Test policies under sensor delay, occlusion, friction change, and human-motion ambiguity.
5. Report recovery behavior separately from first-attempt success.

Among the hidden-variable sources listed above, calibration drift is the one most often omitted from failure analysis. A camera's intrinsic parameters shift with temperature; a joint encoder's zero-point creeps after repeated impacts; a force sensor's bias drifts over an operating shift. None of these changes are directly observable: the robot's software continues to receive numbers, but those numbers no longer mean what they did at startup. In physical terms, calibration drift decouples the coordinate frame assumed by the policy from the coordinate frame actually measured by the hardware, hiding error as confident-looking data rather than as noise.

The mechanism is slow accumulation. Each individual reading stays within sensor tolerance, so no single observation triggers an alarm. The policy anchors its belief state to a subtly wrong world model and compounds that error across every subsequent decision. To detect calibration drift, the system must compare sensor output against a ground-truth reference on a schedule, or trigger recalibration when prediction residuals exceed a threshold. Neither happens automatically without explicit design.

Partial observability becomes useful when it is tied to a closed-loop contract between policy, world, evaluator, and safety constraints. The contract names the hidden-variable inventory, observation stream, belief or confidence field, probing action, timing budget, safety boundary, and result artifact. That is the bridge between a readable concept and a system a skeptical builder can test.

From concept to testable contract

Separate the conceptual claim, the systems claim, and the evidence claim. A good explanation, a clean API, and one successful rollout are different kinds of evidence, and the section should keep them distinct.

Keeping those three kinds of evidence distinct only pays off if the implementation records them in a form a builder can inspect later.

A robust implementation starts with one inspectable baseline whose artifact records observations, actions, units, timestamps, seeds, termination reasons, and the applied perturbation. A maintained-tool version helps only if it preserves that schema and keeps the comparison construct-matched.

1. Write a one-paragraph task contract with observation, action, success, failure, and safety fields.
2. Start with the smallest simulator, dataset, or wrapper that exposes the task contract faithfully.
3. Run one deterministic smoke test and one perturbation test before scaling.
4. Save one artifact containing configuration, seed, metrics, traces, and failure labels.
5. Compare methods only when the same script evaluates the same panel, split, seed set, and metric.

Even with that disciplined baseline in place, rollouts will still fail, and how you read those failures decides whether the artifact earns its keep.

When a partially observable embodied system fails, avoid labeling the whole method as weak. First assign the failure to occlusion, contact sensing, latency, calibration, internal hardware state, other-agent prediction, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.