Section 42.7: Mobile Manipulation: Base, Arm, Perception, And Recovery

"A mobile manipulator is a negotiation between reachability and route planning."

A Whole-Body Systems Notebook

Park a flawless grasp policy two centimeters too far left and it will grope at empty air with total confidence (see Figure 42.7A): in whole-body manipulation, where the base stops silently decides whether arm kinematics, sensor visibility, and the retreat path can ever line up, which is why good mobile manipulators plan navigation and grasping together rather than as isolated modules.

This section folds navigation, mapping, manipulation, and recovery into one application-grade system loop with explicit evidence artifacts.

Theory

Figure 42.7.1 sketches the four-stage loop that organizes this section: sense the scene, stage the base and arm, execute the whole-body task, and recover when contact fails, looping back to sense. The central object is a coupled cost over base pose, arm configuration, visibility, collision risk, and task progress. Local arm planning cannot be optimal if the base pose destroys reachability or line of sight. This joint dependency is called the navigation-manipulation coupling problem, and it is why treating the two as sequential modules reliably fails. By the end of this section you should be able to compute a joint staging score from reachability, view, and risk terms, run the staging algorithm end to end on candidate base poses, and diagnose which of route, stage, perception refresh, arm plan, or retreat branch first broke when a whole-body task fails.

Base staging matters because a robot arm's reachable workspace is a fixed shell in the base frame: a hollow sphere bounded by minimum and maximum extension, with exclusion zones near joint limits. Moving the base two centimeters shifts which part of that shell intersects the target. Consider a 6-DOF (six degrees of freedom) arm with a 60 cm reach. A 5 cm lateral base error can move the target from the center of the reachable shell into a near-singular region where the arm runs out of manipulability (a measure of how freely the arm can move its end effector in every direction from a given joint configuration; near zero manipulability means the arm is close to a singularity). The arm then produces jerky motion, inaccurate contact forces, and failed grasps even though the arm policy is correct. Measured on the BEHAVIOR-1K benchmark (a large-scale simulated suite of household activities used to evaluate embodied agents), a grasp policy trained without base-pose awareness needed roughly 40,000 environment steps to reach 70% success; the same policy with a joint staging reward reached 70% in under 2,000 steps, a twentyfold reduction that the reported result attributes to the base arriving in the manipulability sweet spot rather than anywhere the navigator happened to stop.

How Staging Finds The Pose

Finding that sweet spot reliably, rather than stumbling into it, is exactly what a staging routine is for. Before the base commits to a stop, staging computes the intersection of three sets: the arm's reachable workspace projected into the world frame, the sensor's unoccluded view cone to the target, and the collision-free retreat corridor needed if grasping fails. It scores candidate base poses by how much margin each preserves across all three sets at once. The robot selects the pose with the largest joint margin rather than the one that minimizes travel distance, then refreshes local perception after arriving to catch any map drift before the arm moves.

This coupling is why mobile manipulation is a natural benchmark for embodied AI. It forces a policy or planner to reason across spatial scales and across multiple failure channels in one episode. In reported benchmark studies, systems that plan navigation and grasping in isolation typically fail on roughly 60% of household tasks, while jointly-optimised whole-body planners cut that to under 15% on the same benchmarks (as of 2024): a fourfold reduction that these studies attribute to the single architectural decision to couple staging and grasping rather than to any change in the grasp policy itself.

The joint objective below makes this coupling explicit: the optimal base pose \(q_b^\star\) and arm configuration \(q_a^\star\) minimize a single cost that sums route, reachability, viewpoint, and risk terms together rather than optimizing any one in isolation. The reachability, view, and risk terms were introduced above; the route term \(C_{\text{route}}\) is the cost of the navigation path itself (distance and time to reach a candidate base pose), covered in depth later in this section under multi-timescale reasoning.

$$ (q_b^\star, q_a^\star) = \arg\min_{q_b, q_a} C_{\text{route}}(q_b)+C_{\text{reach}}(q_b,q_a)+C_{\text{view}}(q_b)+C_{\text{risk}}(q_b,q_a) $$

Worked Example

# Score candidate base poses for reachability, view, and risk.
candidates = [
    {"pose": "b1", "reach": 0.92, "view": 0.55, "risk": 0.20},
    {"pose": "b2", "reach": 0.80, "view": 0.90, "risk": 0.10},
    {"pose": "b3", "reach": 0.95, "view": 0.30, "risk": 0.35},
]

scores = []
for c in candidates:
    score = round(0.5 * c["reach"] + 0.4 * c["view"] - 0.6 * c["risk"], 3)
    scores.append((c["pose"], score))

scores.sort(key=lambda row: row[1], reverse=True)
print(scores)

Expected output: The expected ranking prefers the slightly less reachable pose with much better visibility and lower risk. That is often the right whole-body decision in homes, warehouses, and service settings.

Practical Recipe

1. Score base poses with visibility and retreat feasibility, not just arm reachability.
2. Refresh local perception after arriving at the base pose because small route errors matter near contact.
3. Reserve space for retreat and human-safe recovery before the arm starts moving.
4. Log route, base pose, arm plan, and failure branch in one artifact.
5. Benchmark on tasks where the first base pose is intentionally suboptimal so recovery is exercised.

A policy that works in simulation but collapses the moment the base parks two centimeters off is not a manipulation policy; it is a staging dependency waiting to be exposed. Mobile manipulation is a clean example of multi-timescale reasoning. Global route planning runs over meters and seconds, while contact control runs over centimeters and milliseconds. Whole-body success depends on passing the right abstractions between those scales.

Passing abstractions across those scales is only half the challenge; the other half is measuring whether they arrived intact. This makes mobile manipulation a sharp test of coupled evaluation. A navigation benchmark and a grasp benchmark can both score well while the combined system fails, because the interface between them was never optimized as a unit.

When a task fails, ask whether the route, the base pose, the local perception refresh, the arm plan, or the retreat branch first violated its contract. Mobile manipulation only becomes debuggable once those labels stay separate.