Section 26.1: What a skill is; low- vs. high-level actions

"A skill is a promise to the planner: start here, follow this policy, and I will tell you when I am done."

A Hierarchical Policy Designer

Why Hierarchy Matters

Ask a robot to "pick up the cup." Beneath that four-word instruction, roughly 3,000 joint-torque commands per minute must fire in the right order. No human planner thinks in torques. The entire discipline of skill hierarchy exists to let the planner reason in cups while the controller reasons in newtons. Hierarchy separates timing, contact, recovery, and sequencing so a high-level planner can select skills without pretending every low-level policy is deterministic.

Two terms will recur constantly from here on: a low-level action is a single motor command issued on a millisecond timescale (a joint torque or a velocity setpoint), while a high-level skill is a longer-running, self-contained behavior (seconds or more) that decides internally which low-level actions to issue and when to stop. The rest of this section defines the boundary between the two precisely.

The distinction is temporal scale. A low-level action might be a velocity command for 20 milliseconds. A high-level skill might be a 6-second door-opening routine that watches force, pose, contact, and success. Both are actions from the planner's perspective, but only the skill hides a controlled sequence behind a stable, reusable behavioral interface that the perception-action loop can drive without re-deriving every motor command. Figure 26.1.C below lays out this progression from torque command to skill to task on a single timescale.

Formal Contract

Naming that stable interface is only half the story; to use it as a contract the planner can trust, we need to say precisely why hiding the low level is worth the effort and what fields the contract must contain.

Temporal abstraction matters in embodied AI because physical execution is typically irreversible. A gripper that closes on a fragile object cannot undo a 20-millisecond torque command that exceeded safe contact force. Grouping those commands into a skill with an internal force guard lets the skill catch and reverse failures before the planner ever sees a bad state. Without this boundary, a planner operating at the wrong timescale does one of two bad things. It issues unsafe low-level commands directly, or it sequences hundreds of micro-actions and multiplies error propagation at every step. A concrete illustration of the cost: learning a pick-and-place task from scratch at the torque level typically requires on the order of 50,000 demonstration episodes in published benchmarks, while the same task learned at the skill level, where the planner selects from three typed primitives, typically converges in roughly 300; exact counts vary with object variety and controller quality.

The mechanism is a time-extended policy governed by a termination function. At each timestep, the skill's internal policy selects an action; the termination function independently evaluates whether the skill's postcondition, the sensor-space fact that must hold true for the skill to count as successfully finished (for example, the object is in the gripper), is met or a safety guard is violated. The planner waits, issuing no new commands, until termination fires. This suspends the planner's decision cycle for the duration of the skill, reducing its effective branching factor, the number of distinct choices the planner must consider at each decision point, from thousands of torque-space choices to a handful of skill-level options.

The option tuple

Use the option tuple as an audit checklist: initiation states, internal policy, termination probability, and verifier must match the robot task. The equation below states this contract formally: an option \(\omega\) is the triple of an initiation set \(I_\omega\) (the states where the skill may start), an internal policy \(\pi_\omega\) (which action to take), and a termination function \(\beta_\omega\) (the probability of stopping in the current state).

$$\omega=(I_\omega,\pi_\omega,\beta_\omega),\quad a_t\sim\pi_\omega(a\mid s_t),\quad \mathrm{stop}\sim\beta_\omega(s_t).$$

Map the option fields onto behavior trees, task graphs, finite-state machines, or task-and-motion planning nodes so start, act, stop, and verify remain inspectable. Figure 26.1.B shows one such mapping, where a mission goal expands into a task graph of skills with a verifier that can route failures back upstream.

Worked Implementation

The option tuple and its diagram stay abstract until you watch the four fields turn into running code; the fragment below makes the initiation, termination, and verifier fields concrete in a few lines of Python.

Before wiring a skill to ROS 2, BehaviorTree.CPP, Drake, or a learned policy, the code must expose five fields: initiation, progress, termination, verification, and failure reporting. The fragment below shows all five in one dataclass.

# Define a reusable robot skill with explicit start, stop, and verify logic.
# The example separates planner-facing status from low-level control details.
from dataclasses import dataclass

@dataclass
class SkillResult:
    name: str
    status: str
    reason: str

def grasp_ready(state):
    return state["object_visible"] and state["base_distance_m"] < 0.8

def run_grasp_skill(state):
    if not grasp_ready(state):
        return SkillResult("grasp", "blocked", "precondition failed")
    if state["force_n"] >= 8.0 and state["lift_cm"] >= 3.0:
        return SkillResult("grasp", "success", "object verified in gripper")
    return SkillResult("grasp", "retry", "insufficient lift evidence")

print(run_grasp_skill({"object_visible": True, "base_distance_m": 0.5, "force_n": 9.2, "lift_cm": 4.0}))

This expected output means the skill returned a planner-facing verdict, not torque-level behavior. The key field is status='success', which tells the hierarchy that the grasp can hand control back upward because the postcondition was verified in sensor space.

Practical Recipe

1. Name each skill with a verb and object: navigate_to_station, grasp_handle, dock_drone, or change_lane.
2. Write preconditions, postconditions, safety guards, timeout, and recovery behavior before training a policy.
3. Represent sequencing as a finite-state graph, behavior tree, or task-and-motion plan so failures have explicit routes.
4. Use language as a planner only after commands are grounded into a typed skill library with affordance checks.
5. Evaluate composition, not only individual success. Many failures occur when two correct skills meet at a bad boundary.