Section 51.3: Long-horizon tasks

A long-horizon task is a short task that invited all its dependencies to dinner.

A Patient Dependency Graph

A long-horizon task is one whose goal requires many dependent actions executed over an extended time span (seconds to minutes), where early decisions constrain later ones and a single end-of-task reward gives little feedback about which intermediate steps mattered. Figure 51.3A shows this structure as a dependency graph, where each node is a subtask and each edge is a causal precondition the planner must respect. This section builds three capabilities: decompose such a task into verifiable subgoals, build an auditing loop that checks each subgoal's preconditions against live sensing, and repair a broken plan by replanning only the affected suffix rather than restarting. Long-horizon task design becomes useful when it is tied to a named interface, a replayable scenario, a failure diagnostic, and an artifact that records what changed in the action loop.

The key question is practical: Which subgoals can be verified, which failures require backtracking, and what state must persist across the task?

Theory

The defining difficulty of long-horizon tasks is the credit-assignment gap, where credit assignment is the problem of tracing a final outcome back to the specific earlier actions that caused it. A Franka Panda emptying a dishwasher executes roughly 200 to 400 control steps at 30 Hz, yet a sparse benchmark like Meta-World's ML45 (a 45-task suite for evaluating multi-task and meta-reinforcement-learning policies) or the RoboMimic can task (a standard pick-and-place manipulation benchmark used to compare imitation-learning methods) delivers a single success bit at the end. With a discount of \(\gamma = 0.99\), the gradient signal from that terminal reward decays to about \(0.99^{300} \approx 0.05\) by the time it reaches the opening grasp. A flat policy therefore cannot tell which early action mattered. Suffix-replanning architectures fix this by adding explicit subgoal completion signals. Each one converts a single delayed bit into a dense stream of verifiable boundaries that the planner audits at 30 Hz.

Worked Example

Consider setting a table: find plates, clear space, fetch utensils, avoid people, and recover when an object is missing. Each step changes the next observation and the available actions.

To see how those interacting steps play out when one of them breaks, walk through a concrete run with a timer and a blocked drawer.

A blocked drawer, traced step by step

Consider a specific case: a mobile manipulator receives the goal "set a table for four" with a 120-second budget. The planner breaks this into 6 subgoals: (1) locate plates in cabinet A, (2) carry 4 plates to table, (3) locate utensils in drawer B, (4) place utensils, (5) locate glasses, (6) place glasses. At step 3, a chair blocks drawer B. The receding-horizon auditor detects that the precondition "drawer B reachable" is false. This triggers suffix replanning on assumption failure: the auditor invalidates subgoals 3 and 4, inserts a recovery subgoal "move chair," and replans only the remaining suffix. The nominal prefix takes 47 seconds, the repair takes 11 seconds, and the revised suffix takes 31 seconds, finishing at 89 seconds total. Without the subgoal verifier, the agent stalls at step 3 until the time budget expires, reporting failure on all 6 subgoals instead of a success with one logged repair event.

A plan that cannot verify its own progress and repair a broken subgoal is not a plan: it is a wish list with a timer.

Figure 51.3B lays out this same run visually: the locked prefix, the inserted recovery subgoal, the replanned suffix, and how all three fit inside the 120 s budget.

Suffix replanning matters in physical robots because restarting the full plan is often impossible. Plates already placed on the table cannot be "unplaced" cheaply, and the time budget does not reset. Any completed prefix represents real actuator effort and irreversible state change, so discarding it wastes both energy and time. A robot that replans from scratch on every obstacle also moves unpredictably near humans, which raises collision risk. The training cost makes the point concrete in illustrative benchmarks: a reinforcement learning policy that handles mid-task disruptions by restarting typically needs on the order of tens of thousands of environment episodes (roughly 40,000 in one such benchmark) to learn robust recovery, while a policy trained with suffix replanning and subgoal verification can reach a comparable success rate in roughly two orders of magnitude fewer episodes (around 400 in the same benchmark), because it never wastes experience re-learning the completed prefix. Exact ratios vary by task and benchmark, but the qualitative gap is consistent: relearning a completed prefix is wasted experience.

The mechanism splits the plan into a verified prefix and an unverified suffix. At each subgoal boundary the auditor re-checks that subgoal's preconditions against live sensor readings. On a failure it invalidates everything from that index onward, inserts a minimal recovery subgoal, and re-orders only the remaining steps. The prefix stays locked, so completed steps need no re-execution.

The auditing loop above only works if the state it checks actually persists across the task: the subgoal list, each subgoal's verified/pending status, and the timestamp of its last precondition check must survive between control ticks, across a replan, and ideally across a power cycle or human interruption. In practice this state is kept in a small structured log (an ordered list of subgoal records, each with a name, a status, and an elapsed time), the same shape the code example below prints and returns. Without that persisted state, a "repair" is indistinguishable from starting over, because the auditor would have no record of which prefix was already locked in.

# pip install lerobot
# Demonstrates a receding-horizon subgoal auditor for a 6-step manipulation task.
# Mirrors the table-setting scenario: Franka Panda arm, 120-second budget,
# subgoals verified via wrist-mounted force-torque sensor and overhead RGB-D (Red-Green-Blue-Depth) camera.

import time
from dataclasses import dataclass, field
from typing import Callable

@dataclass
class Subgoal:
    name: str
    precondition: Callable[[], bool]   # sensor check before attempting
    execute: Callable[[], bool]        # motion primitive; True = success
    fallback: str = ""                 # name of recovery subgoal if precondition fails
    elapsed: float = 0.0

def run_subgoal_plan(subgoals: list[Subgoal], budget_s: float = 120.0):
    """Receding-horizon audit loop: revalidates preconditions at every subgoal
    boundary and replans only the failed suffix (Sections 51.3 and 7.5)."""
    log = []
    t0 = time.monotonic()
    i = 0
    while i < len(subgoals) and (time.monotonic() - t0) < budget_s:
        sg = subgoals[i]
        if not sg.precondition():
            print(f"[REPLAN] precondition failed: {sg.name!r} -> recovery: {sg.fallback!r}")
            log.append({"event": "replan", "subgoal": sg.name, "recovery": sg.fallback})
            # Insert recovery subgoal immediately before current index
            recovery = next((s for s in subgoals if s.name == sg.fallback), None)
            if recovery:
                subgoals.insert(i, recovery)
            continue   # recheck from recovery subgoal
        t_start = time.monotonic()
        success = sg.execute()
        sg.elapsed = time.monotonic() - t_start
        status = "ok" if success else "fail"
        log.append({"event": status, "subgoal": sg.name, "elapsed_s": round(sg.elapsed, 2)})
        print(f"[{status.upper():4s}] {sg.name} ({sg.elapsed:.2f}s)")
        i += 1
    total = round(time.monotonic() - t0, 2)
    repairs = sum(1 for e in log if e["event"] == "replan")
    print(f"\nTotal: {total}s | Subgoals done: {i}/{len(subgoals)} | Repairs: {repairs}")
    return log

# Stub sensors / primitives for illustration (replace with real ROS 2 service calls)
def drawer_reachable() -> bool:
    return False   # chair is blocking drawer B in this scenario

SUBGOALS = [
    Subgoal("locate_plates_cabinet_A",  precondition=lambda: True,        execute=lambda: True),
    Subgoal("carry_4_plates_to_table",  precondition=lambda: True,        execute=lambda: True),
    Subgoal("locate_utensils_drawer_B", precondition=drawer_reachable,    execute=lambda: True,
            fallback="move_chair"),
    Subgoal("place_utensils",           precondition=lambda: True,        execute=lambda: True),
    Subgoal("locate_glasses",           precondition=lambda: True,        execute=lambda: True),
    Subgoal("place_glasses",            precondition=lambda: True,        execute=lambda: True),
    Subgoal("move_chair",               precondition=lambda: True,        execute=lambda: True),
]

log = run_subgoal_plan(SUBGOALS, budget_s=120.0)

Practical Recipe

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

Long-horizon tasks becomes useful when it is tied to a closed-loop contract for Open-World and Novelty-Robust Embodiment. The contract names the participants, observations, action authority, timing budget, logging artifact, and recovery rule. Without that contract, a system can look capable in a notebook while failing the first time a partner delays, a person corrects it, or a deployment scene changes.

Separate the conceptual claim, the systems claim, and the evidence claim. A plausible mechanism, a clean interface, and a closed-loop result are different claims; the section should keep their evidence separate.

The baseline and maintained-tool version should produce the same artifact schema and run on one task panel. That requirement keeps a systems comparison from becoming a collage of incompatible runs.

1. Write a one-paragraph task contract with observation, action, success, and failure fields.
2. Start with the smallest simulator, dataset, or wrapper that exposes the task contract faithfully.
3. Run one deterministic smoke test and one perturbation test before scaling.
4. Save a single result artifact containing configuration, seed, metrics, videos or traces, and failure labels.
5. Compare methods only when one script evaluates them on the same task panel.

When Long-horizon tasks fails, avoid labeling the whole method as weak. First assign the failure to perception, communication, human input, memory, planning, control, timing, data coverage, safety, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.