Section 12.3: Lifelong and language-conditioned: LIBERO, CALVIN, Meta-World

"A lifelong benchmark does not ask if the agent learned. It asks what the agent forgot, and what it was never given time to relearn."

A Continual Learning Evaluator

What This Section Builds

Imagine a robot that aces drawer-opening on Monday, learns to flip a light switch on Tuesday, and by Wednesday can no longer find the drawer handle it once mastered: that silent collapse is exactly what LIBERO, CALVIN, and Meta-World are engineered to catch. LIBERO tests knowledge transfer across language-conditioned manipulation suites, CALVIN tests long-horizon language-conditioned tabletop behavior, and Meta-World tests multi-task and meta-learning across a structured set of manipulation tasks. As Figure 12.3A suggests, the evidence you need is the whole learning diary, because final success alone cannot show whether earlier skills were retained or quietly lost.

Evaluate learning over distributions, not isolated episodes. That means freezing task order, language templates, held-out goals, adaptation budget, replay budget, seeds, and the rule for measuring forgetting.

Theory

A lifelong benchmark is a sequence of task distributions rather than one static test set. Let \(S_{k,t}\) be success on task family \(k\) after training stage \(t\). A strong final average can hide catastrophic forgetting of earlier skills if earlier tasks lose success after later training, so the evidence artifact should include the whole task-by-stage matrix, not only the last column.

Catastrophic forgetting matters physically because a deployed robot must retain every skill it was trained on. Suppose a Franka arm learns drawer-opening, then trains on a new light-switch task and loses the drawer skill. It fails in the field, and the new training run shows no visible signal of that loss. A task-by-stage matrix catches this failure; a single final-success number does not. Reporting only the last column therefore produces misleading transfer claims.

Figure 12.3B below makes this concrete: it renders the task-by-stage evaluation matrix as a small grid of training and re-evaluation boxes, with orange arrows tracing exactly where success drops as new tasks are trained.

The matrix shows that forgetting happens, but not yet why weights trained on one task degrade on another; the mechanism behind that drop, and the regularization tools used to slow it, are introduced next.

The mechanism is gradient interference. Training on new task data shifts shared weight values to fit the new distribution. Those same weights encoded earlier skills, so earlier skill responses degrade. Think of it like tuning a radio dial: moving to the new station does not erase the old frequency from the spectrum, but the receiver can no longer pick it up because its resonant parameters now point elsewhere. The network has no explicit memory boundary. To anchor the important parameters during each new training stage, you need replay transitions or regularization terms. Elastic Weight Consolidation (EWC) is one such regularizer: it penalizes changes to parameters that mattered most for earlier tasks.

Language conditioning adds another split boundary. A policy can overfit to instruction templates, object names, or goal phrasing while failing to ground new combinations. The evaluation must say whether language, objects, spatial relations, goals, or task families are held out.

Consider a specific case for CALVIN: a policy must hear "push the red block right," complete that subtask, then hear "open the drawer," then "turn on the light." Each instruction is sampled from a held-out template set. If the policy scores 2.1 mean tasks, it succeeds on roughly the first two subtasks before failing at the third. A score of 4.2 means it nearly completes all five. This chain-completion metric reveals failure points that a single-task success rate would not expose, because a policy that memorizes individual instructions can still fail to maintain state across the chain.

To make this matrix-over-final-number argument concrete rather than rhetorical, the next step turns the task-by-stage idea into a few lines of code that any reviewer can rerun.

Worked Example

Code Fragment 1 computes a simple forgetting diagnostic. A method that improves the final task while damaging earlier tasks needs a different claim than a method that retains old skills and transfers to new ones.

# Measure forgetting from a task-by-stage success matrix.
# Rows are task families, columns are training stages after each
# new family has been introduced in the same order for every method.
success_by_stage = {
    "spatial": [0.68, 0.61, 0.55],
    "object": [0.00, 0.64, 0.58],
    "goal": [0.00, 0.00, 0.71],
}

forgetting = {}
for task, scores in success_by_stage.items():
    best_before_final = max(scores[:-1])
    if best_before_final == 0:
        forgetting[task] = "not_previously_trained"
    else:
        forgetting[task] = round(best_before_final - scores[-1], 2)

print(forgetting)

Knowing that the loaders alone will not enforce a fair comparison, the following recipe spells out the protocol you must freeze by hand before any lifelong number is trustworthy.

Practical Recipe

1. State the transfer claim: new objects, new spatial relations, new goals, new task families, longer language-conditioned chains, or faster adaptation.
2. Freeze task order, train/test task split, instruction templates, adaptation budget, replay budget, seeds, and stopping rule.
3. Report the full task-by-stage matrix for lifelong learning, plus final average success.
4. For Meta-World, separate multi-task performance from meta-learning adaptation to held-out tasks.
5. Label failures as language grounding error, task-order forgetting, adaptation overfit, object confusion, goal ambiguity, or controller failure.

A policy that scores well on the final stage but has silently lost its first skill is not a lifelong learner; it is a specialist wearing a borrowed credential.

Lifelong and language-conditioned benchmarks become useful when they expose the sequence of learning, not only the end state. A LIBERO result should show which knowledge shift was tested. A CALVIN result should show how language-conditioned chains were sampled and scored. A Meta-World result should say whether it evaluates multi-task training or adaptation to held-out tasks.

Together, the Concrete Scale callout, the Transfer Benchmark Audit Fields table, and the Practical Recipe above give you a checklist for interpreting a reported split on any of the three suites: which knowledge-shift or chain-length axis it varies, which fields the protocol must freeze, and which forgetting diagnostic accompanies the final number.

Final Competence Versus Learning Dynamics

Separate final competence from learning dynamics. Final success asks whether the policy can solve the last evaluation panel. Forward transfer asks whether earlier training helps later tasks. Backward transfer and forgetting ask whether later training preserves earlier skills. These are different constructs and must be co-computed from one task-by-stage artifact.

A robust transfer benchmark starts with a schedule file listing each training stage, task family, prompt set, replay source, adaptation budget, and evaluation panel. Every method reads the same schedule, tying the comparison to learning dynamics rather than run-specific choices.

1. Write a schedule with task order, prompt templates, train/test tasks, adaptation budget, replay budget, and seeds.
2. Evaluate after each stage on the same held-out panel, not only after final training.
3. Save the task-by-stage success matrix, final average, forward transfer, backward transfer, and forgetting.
4. Stratify results by task family or language condition so one easy family does not dominate the claim.
5. Keep prompt variants and demonstration access identical across methods.

Code Fragment 2 records a transfer schedule. This is the file you want reviewers to inspect before they trust a lifelong-learning number.

# Define one transfer schedule shared by every compared method.
# This schedule fixes task order, adaptation budget, replay budget,
# and held-out evaluation so final averages are construct-matched.
from dataclasses import dataclass, asdict

@dataclass
class TransferSchedule:
    suite: str
    task_order: tuple[str, ...]
    heldout_condition: str
    adaptation_steps: int
    replay_budget: int
    seeds: tuple[int, ...]

    def as_row(self) -> dict[str, object]:
        return asdict(self)

schedule = TransferSchedule(
    suite="LIBERO",
    task_order=("spatial", "object", "goal"),
    heldout_condition="new_language_object_combinations",
    adaptation_steps=0,
    replay_budget=200,
    seeds=(0, 1, 2),
)
print(schedule.as_row())

Expected output: the printed schedule should expose task order, held-out condition, adaptation budget, replay budget, and seeds. If two methods use different schedules, their final success values do not belong in one comparison.

When a lifelong or language-conditioned experiment fails, inspect the matrix before changing the model. A low final average may come from one hard task family, a prompt-grounding failure, forgetting of earlier tasks, insufficient adaptation budget, or a controller failure unrelated to transfer. The failure label should say which one.