Section 59.11: Open-ended research project

"My research question became real when the baseline started winning."

An Open Project With A Fair Test

Your policy beats the baseline by 23 points at midnight, and by lunchtime a single shifted object has erased half that lead: which number do you put in the report, and can you explain the gap? An open-ended capstone is the machinery for answering exactly that. Figure 59.11A lays out the full scaffold, from literature review through hypothesis, experimental design, and the evaluation criteria an instructor uses to assess scientific rigor. The scaffold defines the object of study, connects it to the agent loop, and tests it with a compact implementation. Figure 59.11B distills that scaffold into the compact (Q, B, P, A) loop the rest of this section builds on; that notation is defined formally in the "Formal Object" callout later in this section, so treat every early use of Q, B, P, and A as shorthand for question, baseline, perturbation panel, and artifact bundle until that callout spells out the tuple. Choosing the topic itself, the specific embodiment, task family, and dataset your question will live in, should draw on the failure modes and open problems surveyed across Part XII rather than being invented from scratch; the charter template later in this section is the tool that turns that chosen topic into a gradeable, falsifiable project.

The key question is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

Because the action is the test, the contract that produces the action has to be specified before any model is chosen, which is where the theory of an open-ended project begins.

A result that has never been tested under a controlled perturbation is not yet a finding; it is a hypothesis waiting for its first real challenge.

Make the sensor-to-action interface inspectable before you optimize. For a manipulation task on a Franka Panda arm, save six fields per record. Log the RGB-D frame timestamp, the estimated end-effector pose (robot base frame, metres and quaternion), and the commanded joint torques. Log the measured contact force at the wrist F/T sensor (a force/torque sensor that reports the six-axis wrench applied at the wrist), the control-loop period (nominally 1 kHz on the Panda), and a failure label from the taxonomy in section 3.8. A number without its unit and timestamp is not evidence. It is a guess that cannot be audited after the run.

Worked Example

Keep one concrete rollout in view. A sensor reading becomes an estimate, the estimate constrains an action, the action changes the world, and the next observation confirms or contradicts the assumption. The section's idea is useful only if it improves that loop.

Consider a concrete case: a student hypothesizes that retrieval-augmented memory will cut long-horizon kitchen task failures in Habitat 2.0. The baseline policy (Transformer-BC, where BC denotes Behavioral Cloning, 50 demonstrations) completes 38 of 100 held-out episodes. After adding a nearest-neighbor retrieval step over a 200-episode memory buffer (top-3 retrieved, cosine similarity on goal embedding), completion rises to 61 of 100. The perturbation test shifts the object placement by 0.4 m; completion drops to 44 for the augmented policy versus 29 for the baseline, confirming retrieval helps but does not generalize, which is exactly the gap the capstone must explain. That single table, three numbers per condition, is the evidence artifact. The failure note records 17 episodes where the retrieved memory disagreed with the current scene layout, causing the agent to reach toward an empty counter.

Practical Recipe

Turning that kitchen-task post-mortem into a repeatable habit means fixing the order of operations, so that every future project produces an auditable artifact rather than an unexplained number.

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

Perturbation tests matter because physical deployment never matches the training distribution. A policy that scores well in a fixed simulator may be exploiting room layout, lighting, or object placement as implicit cues, and on a real robot those cues shift every session as furniture moves, light changes, and calibration drifts. Without a perturbation test, you cannot tell a genuine capability gain from an artifact of the evaluation setup.

A perturbation test re-runs the same episode set under one controlled change at a time. Shift object placement by a fixed offset, add Gaussian noise to depth readings, or introduce a fixed observation delay. Then compute the metric drop relative to the unperturbed baseline. A method whose advantage shrinks by more than half under a single perturbation is over-fitted to the nominal condition; a method whose advantage holds shows genuine robustness. In the kitchen task example above, the retrieval policy's advantage over the baseline was 23 percentage points under nominal conditions and only 15 under a 0.4 m placement shift. That 35% collapse from a single centimeter-scale change is why a perturbation test transforms a hypothesis into evidence. The perturbation must be applied identically to both the baseline and the intervention to keep the comparison valid.

def validate_charter(payload: dict[str, object]) -> dict[str, object]:
    assert payload, "payload must not be empty"
    return payload

# Open-ended project charter.
charter = {
    "question": "Does retrieval-augmented policy memory reduce long-horizon kitchen failures?",
    "baseline": "same policy without retrieval memory",
    "perturbation_panel": ["delayed observations", "object moved mid-task"],
    "artifact_bundle": ["config", "metrics", "replay", "failure_note"],
}
print(validate_charter(charter))

{'question': 'Does retrieval-augmented policy memory reduce long-horizon kitchen failures?', 'baseline': 'same policy without retrieval memory', 'perturbation_panel': ['delayed observations', 'object moved mid-task'], 'artifact_bundle': ['config', 'metrics', 'replay', 'failure_note']}