A forecast without uncertainty is just a confident guess with better typography.
A Horizon-Aware Predictor
This section builds on the error-accumulation dynamics introduced in section 36.3, where compounding prediction errors motivate the need to bound model trust over a horizon. The uncertainty representations developed here are put to direct use in section 37.2, which shows how ensemble disagreement and probabilistic rollouts drive planning in model-based RL and MPC (model predictive control, which replans an action sequence at every step from current state). The same aleatoric/epistemic split recurs in Part XI alongside safety constraints in Chapter 54, where uncertainty thresholds gate deployment decisions.
A robot arm reaches for a cup on a cluttered desk. Its world model predicts the cup's position with high confidence one step ahead, but five steps out the prediction fan has exploded: will the cup slide? Will the fingers contact the rim or miss? A single point estimate cannot answer that. Modern embodied systems fail not because their average predictions are wrong, but because they act as if every prediction is equally trustworthy. Right now, as robots move from lab benches into hospitals and warehouses, knowing when not to trust your own model is the capability that separates safe deployment from silent failure. In this section you will distinguish aleatoric from epistemic uncertainty, implement ensemble-based disagreement signals, and connect calibrated uncertainty estimates directly to planning decisions that change robot behavior.
The practical job of uncertainty is to change action selection or trigger a fallback. If uncertainty never alters behavior, it is only reporting, not decision support.
Figure 36.4B illustrates the widening confidence band described above. The application of learned dynamics models to model-based RL and MPC is in Section 37.2. This section focuses on prediction accuracy, uncertainty estimation, and how ensemble disagreement signals model reliability limits.
Aleatoric Versus Epistemic Uncertainty
A predictive model can report a distribution over next states, for example, where \(\mathcal{N}\) denotes a Gaussian (normal) distribution with mean \(\mu_\theta\) and covariance \(\Sigma_\theta\) (the spread term), both predicted by a network with parameters \(\theta\) from the current state \(s_t\) and action \(a_t\):
$$ p_\theta(s_{t+1}\mid s_t, a_t) = \mathcal{N}(\mu_\theta(s_t,a_t), \Sigma_\theta(s_t,a_t)). $$
The covariance may reflect irreducible environment noise, while disagreement across model ensemble members estimates epistemic uncertainty from limited or out-of-support data. A planner should react differently to the two: aleatoric noise may require robust costs, while epistemic uncertainty often calls for caution, exploration, or fallback control.
The distinction matters operationally. Aleatoric uncertainty usually remains high even after more data, because it belongs to the task itself: deformable packages vary, human partners move unpredictably, and wet floors slip. Epistemic uncertainty should shrink when the robot gathers matched data from the problematic regime. Chua et al.'s PETS exploits exactly this mechanism on MuJoCo Half-Cheetah. Early in training the bootstrap ensemble disagrees sharply about fast, unexplored gaits, so the CEM planner penalizes them. After roughly 100 real rollouts the ensemble spread in those gaits collapses as the members converge on matched data. The residual variance in foot-ground contact stays put, because that part is genuine aleatoric noise rather than ignorance. If a system keeps reporting high epistemic uncertainty after many demonstrations, that often points to poor state representation, stale calibration, or a model family that cannot express the relevant mode switch.
| Uncertainty type | Typical cause | Planner response |
|---|---|---|
| Aleatoric | Stochastic contact, noisy sensing, human motion | Optimize expected or risk-sensitive cost over the noise |
| Epistemic | Little data, unseen states, model misspecification | Reduce trust, shorten horizon, gather data, or invoke a safe fallback |
Worked Probe
Deciding between those two planner responses first requires a concrete number for disagreement, so the next probe extracts exactly the ensemble spread that would gate the choice.
This probe compares the mean and spread of a tiny ensemble of one-step predictions. It is not a full uncertainty method, but it exposes the exact statistic the planner would need to gate trust.
# Estimate ensemble mean and disagreement for a one-step rollout.
from statistics import mean, pstdev
ensemble_predictions = [0.48, 0.50, 0.51, 0.63]
mu = round(mean(ensemble_predictions), 3)
sigma = round(pstdev(ensemble_predictions), 3)
print({"ensemble_mean": mu, "ensemble_std": sigma, "members": ensemble_predictions}){'ensemble_mean': 0.53, 'ensemble_std': 0.06, 'members': [0.48, 0.5, 0.51, 0.63]}
Read the uncertainty output as a trigger for risk-aware action: high epistemic uncertainty should widen safety margins, ask for information-gathering behavior, or reject a plan whose predicted success depends on unknown dynamics.
statistics module. Three members agree tightly while one drifts. In a real planner, that disagreement is a signal to reduce confidence in the imagined future even if the mean still looks plausible.Step-Through: ensemble disagreement gating a plan
Trace a tiny two-step rollout where a 3-member ensemble predicts a scalar state and the planner gates trust on the disagreement. Take a calibration threshold of std > 0.10 meaning "back off".
Step 1 (horizon t+1). Members predict 0.50, 0.52, 0.49. Mean = (0.50 + 0.52 + 0.49) / 3 = 0.503. Population std: deviations are -0.003, +0.017, -0.013; squared = 0.000009, 0.000289, 0.000169; sum = 0.000467; /3 = 0.000156; sqrt = 0.012. Std 0.012 < 0.10, so the planner trusts this step and keeps the action.
Step 2 (horizon t+2). Feeding each member's t+1 output forward, they now predict 0.61, 0.78, 0.55 (one member extrapolates hard). Mean = (0.61 + 0.78 + 0.55) / 3 = 0.647. Deviations: -0.037, +0.133, -0.097; squared = 0.00137, 0.01769, 0.00941; sum = 0.02847; /3 = 0.00949; sqrt = 0.097. Std 0.097 is just under 0.10, a borderline case.
Decision. Disagreement grew 8x from t+1 to t+2 (0.012 to 0.097). Even though the mean (0.647) looks like a plausible next state, the planner shortens its trusted horizon to a single step and replans at t+1 rather than committing to the two-step plan. The lesson: the mean alone (smooth 0.503 to 0.647) hides the fan-out that the std exposes.
Chua et al.'s PETS (Probabilistic Ensembles with Trajectory Sampling, 2018) illustrates how ensemble uncertainty changes real behavior. PETS trains a bootstrap ensemble of four probabilistic neural networks on MuJoCo's Half-Cheetah task. During planning, the CEM (Cross-Entropy Method) optimizer samples action sequences and propagates each sequence through a randomly selected member at every step, so the spread across members inflates the predicted cost whenever the imagined trajectory enters low-data regions. In practice, this makes the planner avoid fast, jerky motions early in training (epistemic uncertainty is high there) while accepting the irreducible variability in foot-ground contact (aleatoric noise). The robot learns effective locomotion in roughly 100 real rollouts, far fewer than model-free baselines, precisely because the uncertainty signal is wired into the cost rather than only logged.
Use ensemble bootstraps, probabilistic heads, or calibrated dropout only if the planning loop consumes their output. Save coverage metrics, negative log-likelihood, and safety-trigger counts alongside raw error so calibration can be audited later. PyTorch and JAX make the modeling easy; the hard part is plumbing the uncertainty into Nav2, MoveIt 2, or an MPC safety gate so high uncertainty actually changes behavior.
Calibration And Failure Modes
Wiring uncertainty into the planner only helps if that uncertainty is trustworthy, and the estimates themselves fail in specific, recurring ways that a careful engineer must anticipate.
The most common failure is confident extrapolation. All ensemble members may share the same narrow training regime and therefore the same blind spot, a pattern we call the shared blind-spot collapse. The planner sees narrow intervals, treats them as reliable, and acts boldly on a trajectory every model gets equally wrong. A robot in this state applies full joint torque or commits to a fast grasp exactly when the model's flaw is most exposed. The narrow interval suppresses the safety gate that would otherwise slow the motion or request confirmation. In practice, this collapse has been implicated in arm collisions during handovers, and it typically produces overconfident foot placement on surfaces the training distribution never included, though the exact contribution of shared-blind-spot collapse versus other causes in any single reported incident is rarely isolated. The cause is mechanical. A bootstrap ensemble (a set of models each trained on a different random resample of the same dataset, so their disagreement is meant to reveal where data is thin) shares the same feature extractor and diversifies only through data subsampling, so in sparse regions of state space every member converges to a similar extrapolation. Their variance then reflects sampling noise, not genuine disagreement about the dynamics.
Think of the shared blind-spot collapse like a group of cooks who all trained in the same kitchen: ask them to taste an unfamiliar spice and they will all guess "paprika" with high confidence, because none of them has ever encountered cardamom. Their agreement does not mean they are right; it means they share the same gap in experience. An ensemble of neural networks trained on the same narrow operating regime behaves exactly this way: in states the training data never covered, every member extrapolates in the same direction and their variance collapses to near zero, making the planner feel certain precisely where it should be most cautious.
Failure Mode: Stale Uncertainty
A second failure is temporal mismatch. The uncertainty estimate is computed for the latent state before a new observation arrives, yet the controller treats it as describing the current physical state. In contact-rich tasks, that single stale frame turns a cautious policy brittle. The stakes can be significant: in illustrative legged-locomotion benchmarks, policies acting on one-step-stale uncertainty have been observed to fall on uneven terrain several times more often than otherwise-identical policies that recompute uncertainty after each observation; the exact multiplier is benchmark-dependent rather than a fixed constant.
A model that cannot say "I do not know" is not humble about uncertainty; it is simply unaware of its own limits.
A practical calibration panel should therefore include both nominal and shifted conditions: new object materials, lighting changes for vision-conditioned models, altered contact friction, and delayed observations. If the interval coverage collapses under those matched perturbations, the uncertainty estimate is not yet a reliable control signal.
For each horizon, compare predicted interval coverage (the fraction of held-out actual outcomes that fall inside the predicted interval, which should match the interval's stated confidence level) with empirical coverage on held-out rollouts. If the model says 90 percent intervals but covers only 50 percent of actual next states, the planner should treat those intervals as fiction.
Checkpoint
So far: uncertainty estimates can fail through shared blind-spot collapse (all ensemble members miss the same way), through staleness (the estimate describes an outdated state), or through simple miscalibration (stated confidence does not match observed coverage); the calibration rule above gives a concrete test for the third failure, and the tools below show how to fix it.
The netcal Python library's IsotonicRegression calibrator (a non-parametric method that learns a monotonic correction mapping from predicted confidence to observed accuracy) can post-hoc recalibrate a miscalibrated ensemble without retraining: fit it on a small held-out rollout set and apply it before the intervals reach the planner. A quick sanity check is to plot a reliability diagram (a plot comparing predicted confidence against observed accuracy, where a well-calibrated model traces the diagonal) using netcal.presentation.ReliabilityDiagram across horizons 1, 3, and 5 before wiring any uncertainty signal into a cost function. If the diagram shows consistent over-confidence at horizon 3 or beyond, do not extend the planning horizon until the calibrator is retrained on data from that regime. Applying a calibrator trained only on short-horizon rollouts to longer horizons is a common silent failure that produces narrow intervals exactly where they should widen.
A common assumption is that all prediction uncertainty is reducible: collect more data, train longer, and the model will eventually become certain. This is wrong in embodied AI because aleatoric uncertainty is irreducible by design. Stochastic contact forces, unpredictable human motion, and sensor noise are properties of the physical world, not of the model's ignorance. Gathering more data lowers epistemic uncertainty in regions the robot has visited, but it cannot eliminate the genuine randomness in the environment. The correct mental model is to treat the two types separately: pursue data collection to close the epistemic gap, then design robust costs or safety margins to handle the aleatoric floor that remains.
Uncertainty estimates can become overconfident exactly where they are most needed, namely on out-of-distribution states. Never assume that a narrow interval means safety unless coverage was verified on a matched perturbation panel.
A quadruped stepping on mixed terrain may face genuine aleatoric slip variability, while a warehouse arm asked to manipulate a never-seen deformable package faces epistemic uncertainty. The first calls for robust contact costs; the second may justify slowing down, gathering data, or asking a human to intervene.
Real-World Application: autonomous driving prediction stacks
Waymo's behavior prediction system outputs multimodal trajectory distributions with per-mode confidence rather than single point forecasts, so the planner can widen its safety envelope when a cyclist's intent is ambiguous (high epistemic uncertainty at an unusual intersection) versus a car cruising in lane (low uncertainty). The downstream planner explicitly reserves more clearance for high-variance predicted agents, the same uncertainty-to-action wiring this section argues for, just at city scale.
Lab: watching ensemble disagreement grow with horizon
Goal. See for yourself that epistemic uncertainty (ensemble spread) explodes on out-of-distribution states while staying tight on in-distribution ones, and that the mean prediction hides this.
Tools needed. Python with gymnasium, numpy, torch, and matplotlib. Use the Pendulum-v1 environment (continuous state, cheap to roll out).
Steps (15-30 min). (1) Collect roughly 5,000 transitions by acting with random torque, but only in the upright half of the state space (cos(theta) > 0), creating a deliberate data gap. (2) Train a bootstrap ensemble of five small MLPs (two hidden layers, 64 units) to predict the next state, each on a different resample of the data. (3) Roll out a 10-step open-loop prediction from a start state by feeding the mean prediction forward, recording the per-step standard deviation across the five members.
What to vary. Start the rollout once from an in-distribution upright state and once from a held-out downward state (cos(theta) < 0) the ensemble never saw. Also vary ensemble size (3, 5, 10) and the size of the data gap.
What to observe. Plot ensemble std versus horizon for both start states. The in-distribution curve should stay low and grow slowly; the out-of-distribution curve should fan out fast. Note whether the means look equally smooth in both cases (they usually do), demonstrating that the spread, not the mean, is the signal a planner must gate on. As a stretch, threshold the std and print at which horizon each rollout would trigger a "back off" decision.
This section pairs naturally with Chapter 54 on safety, the ensemble modeling in Section 37.2, and state-estimation noise models in Chapter 8.
Conformal prediction for robot safety guarantees (2024). Conformal prediction (a distribution-free statistical method that wraps any predictor to produce prediction sets with a guaranteed coverage probability, without assuming a Gaussian or other parametric error model) is replacing heuristic thresholds in high-stakes manipulation. Lindemann et al. (2024, "Conformal Prediction for Robotics," IEEE ICRA) demonstrate that conformal risk control applied to a learned dynamics model yields provably valid prediction sets at any user-specified coverage level without Gaussian assumptions, enabling hard safety constraints rather than soft probabilistic suggestions.
Scalable epistemic uncertainty in large vision-language policies (2024-2025). As embodied policies move toward VLA (vision-language-action) architectures, classical ensembles become prohibitively expensive. Google DeepMind's uncertainty-aware extensions of RT-2X (2024, as of preprint) and Carnegie Mellon's work on token-level Laplace approximations for action-chunk policies suggest that lightweight post-hoc calibration of frozen transformer backbones can recover meaningful epistemic signals without retraining, gating high-uncertainty action tokens before execution.
Uncertainty-driven active data collection in contact-rich manipulation (2025). Rather than passively logging uncertainty events, recent work from Berkeley's RAIL lab (2025, "ORCA: Online Robot Curriculum via Uncertainty Allocation") routes ensemble disagreement signals directly into a curriculum scheduler that prioritizes new demonstrations from the highest-epistemic-uncertainty state clusters. This closes the identification cycle: the same signal that gates the planner also directs the data-collection policy, shrinking epistemic blind spots without human intervention.
Open problem for PhD students. Current conformal prediction approaches for dynamics models require an exchangeability assumption that breaks down under closed-loop feedback: the robot's own actions change the distribution of future states, so held-out calibration data is no longer representative of rollout-time distributions. Designing adaptive conformal sets that remain valid under policy-induced distribution shift, while staying computationally cheap enough for online replanning, is an open and tractable thesis problem with direct deployment consequences for contact-rich and legged-locomotion domains.
Can you name a setting where high aleatoric uncertainty should not automatically stop the robot, and a setting where high epistemic uncertainty probably should?
Prediction error says, "I was wrong." Uncertainty says, "I might be wrong, so plan accordingly."
Good uncertainty does not merely decorate a forecast. It changes which futures the planner trusts, which actions it chooses, and when it should back off.
Choose one embodied task and define a calibration panel for it. What would count as acceptable interval coverage at horizons 1, 3, and 5?
Project Ideas
Beginner (weekend): Ensemble disagreement visualizer in Gymnasium. Train a small bootstrap ensemble of three neural networks to predict next states in CartPole-v1 using Gymnasium, then plot how ensemble standard deviation grows with rollout horizon. The key challenge is distinguishing genuine disagreement from numerical noise when all members see the same narrow training distribution.
Intermediate (1-2 weeks): Uncertainty-gated MPC in MuJoCo. Implement a simple PETS-style planner on HalfCheetah-v4 in MuJoCo where the CEM optimizer multiplies predicted cost by ensemble disagreement so high-uncertainty action sequences are penalized. The key challenge is keeping the replanning loop fast enough (under 100 ms per step) while sampling from four probabilistic network heads and propagating that signal correctly into the cost without collapsing the diversity that makes the disagreement meaningful.
Intermediate (1-2 weeks): Calibration panel for a LeRobot manipulation policy. Use LeRobot's pretrained ACT policy on a pick-and-place task and build a held-out perturbation panel (new object colors, added table clutter, slight camera angle shifts) to measure whether the policy's built-in uncertainty proxies (action-chunk variance) remain calibrated under distribution shift. The key challenge is defining matched perturbations that isolate epistemic from aleatoric spread rather than conflating the two.
Bibliography & Further Reading
Hansen, N. et al.. "TD-MPC2: Scalable, Robust World Models for Continuous Control." (2023). https://arxiv.org/abs/2310.16828
A modern latent model-based baseline that readers should compare against when thinking about uncertainty-aware planning.
Chua, K. et al.. "Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models." (2018). https://arxiv.org/abs/1805.12114
PETS is a canonical uncertainty-aware ensemble method for model-based RL.
Deisenroth, M., and Rasmussen, C.. "PILCO: A Model-Based and Data-Efficient Approach to Policy Search." (2011). https://dl.acm.org/doi/10.5555/3104482.3104583
PILCO remains a useful reference for uncertainty propagation under data scarcity.