Section 3.2: Classical modular robotics pipeline

"Each module trusts that the previous one did its job. The pipeline is only as honest as its cleanest interface."

A Modular Pipeline, Third Stage

Three modules can each pass every unit test, sit in the same robot, and still drive it into a wall, because the bug lives not in any module but in the silent space between them. Figure 3.2A shows the structure this section dissects: a perception module writes to a world model, a planner queries that model, and a controller tracks the plan, with a distinct failure mode lurking at each handoff. The section defines the object of study, connects it to the agent loop, then tests it with a compact implementation.

Figure 3.2 reframes the same pipeline as a closed-loop evidence, decision, consequence pattern, the lens that exposes where interfaces and handoffs can fail. This is the same sense-perceive-plan-act stack covered in Section 3.1, now examined at its seams rather than its stages. The key question is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

The practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

A classical modular pipeline exists because robotics teams need inspectable boundaries when perception, localization, mapping, planning, and control are owned by different algorithms or teams. Concretely, the pipeline is a fixed sequence of named stages, each with its own module boundary: a perception stage turns raw sensor data into an estimated pose or map feature, a localization/mapping stage fuses that estimate into a consistent world model, a planning stage queries the world model to produce a trajectory or corridor, and a control stage tracks that trajectory with actuator commands. The rest of this section shows how to define, check, and instrument the contract at each of those boundaries, which is the practical skill this section is building toward. The contract for a module is not only a data type. It also includes coordinate frame, timestamp, uncertainty, rate, preconditions, and fallback behavior.

A perception module should not publish only "block at \((0.42, 0.18, 0.03)\)." A useful handoff carries \((\text{pose}, \Sigma, \text{frame}, t, \text{confidence})\), where \(\Sigma\) (a covariance matrix quantifying how spread out the estimate's error is expected to be) is the pose uncertainty. The frame and timestamp make the estimate usable by a planner. The pipeline relies on conditional independence, meaning that once a module's declared output is fixed, its downstream consumers depend on nothing else inside it: each module can improve behind its interface as long as its output contract stays stable. The rest of this section defines this failure mode precisely as interface optimism and then shows, with a runnable acceptance gate, exactly which fields catch it. A physical robot cannot pause time while modules negotiate. Each stage commits to an output and passes it downstream before the world changes. Suppose perception takes 80 ms and planning takes 40 ms. The controller then acts on a world already 120 ms old. At a modest 0.5 m/s walking pace, that 120 ms window moves the robot 6 cm before it can react. That distance roughly matches a doorframe gap, and it is exactly the margin that makes autonomous wheelchairs clip doorframes even when every module reports success. The robot's own motion during that window compounds the error, and no single module can correct it alone. The mechanism is a staged queue (a pipeline of buffers, one per module boundary, that holds each stage's output until the next stage is ready to read it): each module reads from an input buffer, transforms its representation, writes to an output buffer, and clears its internal state before the next cycle. Conditional independence holds when a module's transformation depends only on the buffer contents at the declared interface, not on another module's internal state. When that assumption breaks, a planner can cache an old map while perception has already updated it. Both modules behave correctly in isolation, but their combined output is wrong. The failure mode is interface optimism: every module is correct under its own assumptions, but those assumptions cannot all hold in the real episode. This is a structural consequence of partial observability.

In representative logistics deployments (as of 2024), adding a four-field contract check (frame, timestamp, covariance, confidence) has typically reduced silent handoff failures by roughly an order of magnitude with no changes to any perception or planning algorithm, though the exact reduction depends on the deployment's sensor mix and prior contract discipline.

Interface optimism is not a textbook abstraction; it has stalled real vehicles in the field, and the most instructive case is one where the contract was missing the single field that would have caught it. Consider a specific case: the 2007 DARPA Urban Challenge winner, Boss (Carnegie Mellon), used exactly this structure. A Velodyne lidar perception module published obstacle poses in a world frame at 10 Hz. A route-planning module consumed those poses and generated a drivable corridor (a bounded strip of free space around a candidate path that the controller is allowed to track). A controller tracked the corridor at 100 Hz. During sensor recalibration, a coordinate mismatch slipped in, and one obstacle pose arrived in the vehicle frame instead of the world frame. The planner silently generated a corridor offset by several meters, and Boss drove toward a curb. The fix did not touch perception or planning. Engineers added a frame-id check at the interface, exactly the contract enforcement the code fragment below demonstrates. Reports on production vehicle stacks, including Waymo's early stack, describe similar timestamp and uncertainty gates being added after comparable field incidents, though the public record does not confirm this was in direct response to the same failure pattern.

Worked Example

Having traced the acceptance algorithm field by field on paper, the next step commits it to runnable code so the same four messages can be dropped or forwarded by an actual gate.

The point of the modular pipeline is that a message is a contract, not a number. The example below defines a perception message that carries (pose, covariance, frame, stamp, confidence), and a planner that refuses any message whose uncertainty is too high or whose timestamp is too old. This is the difference between a pipeline that fails loudly at the interface and one that silently consumes stale evidence.

from dataclasses import dataclass

NOW = 10.00          # current planner clock (seconds)
MAX_STALE = 0.20     # reject evidence older than 200 ms
MAX_SIGMA = 0.05     # reject pose with std-dev above 5 cm

@dataclass
class PerceptMsg:
    pose: float       # x in the map frame (m)
    sigma: float      # pose std-dev (m)
    frame: str        # coordinate frame id
    stamp: float      # time the percept was valid (s)
    confidence: float # detector score in [0, 1]

def planner_accepts(m: PerceptMsg):
    if m.frame != "map":
        return False, f"wrong frame: {m.frame}"
    if NOW - m.stamp > MAX_STALE:
        return False, f"stale by {NOW - m.stamp:.2f}s"
    if m.sigma > MAX_SIGMA:
        return False, f"uncertain: sigma={m.sigma:.3f}m"
    return True, "accepted"

inbox = [
    PerceptMsg(0.42, 0.02, "map",      9.95, 0.91),  # good
    PerceptMsg(0.42, 0.02, "base_link", 9.98, 0.91), # wrong frame
    PerceptMsg(0.42, 0.02, "map",      9.40, 0.91),  # stale
    PerceptMsg(0.42, 0.11, "map",      9.96, 0.55),  # too uncertain
]
for m in inbox:
    ok, why = planner_accepts(m)
    print(f"{'PLAN' if ok else 'DROP'}: {why}")

Expected output: only the first message reaches the planner; the other three are dropped with a specific reason. This is "interface optimism" caught at the boundary: each upstream module believed its pose was correct, but the contract reveals that frame, age, and uncertainty disqualify three of the four. Change MAX_STALE or MAX_SIGMA and watch which messages flip, which is the calibration question every real pipeline must answer.

Practical Recipe

1. Write the module contract before writing any code: specify sensor type and rate (e.g., Velodyne VLP-16 at 10 Hz or Intel RealSense D435 at 30 Hz), coordinate frame (robot base or world map), maximum acceptable staleness, and the covariance threshold at which a downstream planner's path deviates by more than one robot footprint. For a 0.6 m wide autonomous mobile robot (AMR) navigating at 0.8 m/s, that threshold is typically under 8 cm (1-sigma).
2. Build a minimal pipeline that logs the full five-field message (pose, covariance, frame, timestamp, confidence) to disk before adding any planning logic. On a Franka Panda arm, this means verifying that the wrist camera publishes in the robot base frame, not the camera optical frame, before the first pick attempt.
3. Add a contract-checking gate (frame, staleness, covariance) before connecting perception output to the planner. Measure the rejection rate over 200 cycles in simulation (MuJoCo or Isaac Sim) to calibrate thresholds under realistic sensor noise.
4. Record failures with structured labels: frame-mismatch, stale-evidence, high-uncertainty, plan-infeasible, or controller-saturation. On a Boston Dynamics Spot deployment, controller-saturation labels from steep-terrain episodes pointed directly to a missing terrain-slope precondition in the planning module's interface contract.
5. Run at least one perturbation test that degrades one module in isolation: drop 30 percent of lidar returns to simulate rain or dust, or inject 80 ms of artificial latency on the perception channel to exceed the staleness gate. If the pipeline fails silently rather than logging a labeled rejection, the contract enforcement is incomplete.

A pipeline that passes every unit test but fails in the field has not failed at a component; it has failed at a contract.

The modular pipeline earns its keep only when a closed-loop contract binds perception, estimation, planning, and control into one system. That contract names the observation stream, the action representation, the timing budget, the safety boundary, and the result artifact. It is the bridge from a readable concept to a system a skeptical builder can test.

Separate the conceptual claim, the systems claim, and the evidence claim. A good explanation, a clean API, and one successful rollout are different kinds of evidence, and the section should keep them distinct.

A robust implementation starts with one inspectable baseline whose artifact records observations, actions, units, timestamps, seeds, termination reasons, and the perturbation applied. The maintained-tool version is useful only if it preserves that schema and lets the comparison remain construct-matched.

Before moving on, ask yourself: if your pipeline passes every unit test but the robot still drives into a wall, which interface log would you check first, and what label would you expect to find there?

1. Write a one-paragraph task contract with observation, action, success, failure, and safety fields.
2. Start with the smallest simulator, dataset, or wrapper that exposes the task contract faithfully.
3. Run one deterministic smoke test and one perturbation test before scaling.
4. Save one artifact containing configuration, seed, metrics, traces, and failure labels.
5. Compare methods only when the same script evaluates the same panel, split, seed set, and metric.

When Classical modular robotics pipeline fails, avoid labeling the whole method as weak. First assign the failure to perception, state estimation, planning, control, timing, data coverage, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.

The strongest modular postmortem asks, "Which interface accepted a value it should have rejected?" A localization drift case might still produce a valid pose message, but the covariance or timestamp should reveal that the planner is consuming stale evidence. A planner infeasibility case should report whether no path exists, the map is inconsistent, or the controller cannot satisfy the requested curvature. These distinctions make modular systems slower to assemble but faster to debug.

pip install numpy pandas

# Complete compact evidence trace for the section lab.
# Extend these records with values produced by your actual environment or simulator.
import pandas as pd

records = [
    {"run": "baseline", "seed": 0, "success": 0.72, "failure_label": "none"},
    {"run": "library_shortcut", "seed": 0, "success": 0.78, "failure_label": "none"},
    {"run": "baseline_perturbed", "seed": 0, "success": 0.54, "failure_label": "latency"},
]
print(pd.DataFrame(records))