Section 60.1: One-semester graduate course (14 weeks)

"Fourteen weeks is plenty of time, provided every theorem eventually touches a log file."

A Graduate Plan With A Simulator

Fourteen weeks is roughly 98 days, and on day one a graduate student who has never written a Kalman filter must, by the final demo, defend a policy that closes the loop on real hardware: the entire challenge of this schedule is deciding what enters each of those weeks so the arc holds. First we define the object of study, then we connect it to the agent loop, then we test it with a compact implementation. Figure 60.1A lays out the full schedule as a Gantt chart, showing how the four phases (foundations, simulation and RL, language-vision-action, evaluation and capstone) tile the 14 weeks and where the mid-term and project milestones fall.

The key question is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

Answering those four questions in a way a grader can check turns directly into a design rule for the interface itself.

The practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

Worked Example

Keep one concrete rollout in view, because the entire 14-week schedule is engineered to make this loop legible by the final week. In a Week 5 MuJoCo lab, a joint-encoder reading becomes an end-effector pose estimate, the estimate feeds a goal-conditioned policy that emits a delta command, the FetchReach gripper moves, and the next observation either lands within the 5 cm success tolerance or reveals a drift the student must diagnose. By Week 12 the same loop runs on an Isaac Lab Franka Panda with a SayCan-style planner sitting on top, but the contract is identical: sense, estimate, act, verify. A syllabus is well sequenced only when every later week reuses the observation-action-metric-failure structure students first wrote in Week 5 rather than introducing a fresh, unrelated abstraction.

The FetchReach loop, graded

Before this loop can run, students need the state-estimation and reward-design vocabulary it depends on: a goal-conditioned policy (a policy that takes the target state as an extra input rather than being retrained per goal) and a success metric defined in terms of an observable estimate, both of which are introduced in the foundations weeks (1 to 3) referenced in the pathway callout above. The worked example below assumes that vocabulary is already in place.

Consider a specific case. In Week 5 of the schedule below, students implement a goal-conditioned policy in MuJoCo's FetchReach-v2 environment. The observation vector holds 10 dimensions (end-effector position, velocity, and goal coordinates), the action is a 4-DOF delta command, and success means the end-effector stays within 5 cm of the goal for 5 consecutive steps. A baseline PD controller (a proportional-derivative feedback law that drives the end-effector toward the goal using position error and its rate of change) reaches roughly 40% success in 50 episodes. (The baseline exists so students have a simple, fully inspectable reference; every later, more capable policy in the course is judged against how much it improves over this number.) One imitation-learning pass (training a policy directly on recorded human demonstrations, rather than reward-driven trial and error) on 200 teleoperated demonstrations lifts the success rate to 78%. The graded artifact holds both runs, the metric panel, and at least one failure trajectory annotated with the error type (state estimation error vs. planning error vs. control error). This concrete accounting forces students to confront the gap between a policy that looks smooth in replay and one that closes the loop reliably. A policy that passes every simulator metric but fails on the first real hardware trial is not a trained policy: it is a well-rehearsed approximation of one.

To make this concrete, the syllabus table below (Suggested Weekly Arc for Graduate Teaching) shows exactly which weeks carry which content, how the algorithm in the next section builds a full 14-week sequence phase by phase, and how the named anchors (Drake, D4RL, OpenVLA, SayCan) turn each phase into a runnable assignment rather than an abstract label; instructors can copy that table directly into a syllabus template.

Practical Recipe

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

def validate_plan(payload: dict[str, object]) -> dict[str, object]:
    assert payload, "payload must not be empty"
    return payload

# Graduate-course planning card.
plan = {
    "weeks": 14,
    "artifacts_per_week": 1,
    "paper_discussion_every_week": True,
    "capstone_milestones": ["proposal", "baseline", "failure_review", "final_demo"],
}
print(validate_plan(plan))

{'weeks': 14, 'artifacts_per_week': 1, 'paper_discussion_every_week': True, 'capstone_milestones': ['proposal', 'baseline', 'failure_review', 'final_demo']}