Section 17.3: Isaac Lab with SKRL / rl_games / RSL-RL

"Isaac Lab sets the stage, randomizes the props, and rings the bell. What the agent does next is the trainer's problem."

A Careful Control Loop

Two teams train the identical Anymal-D rough-terrain policy, swap nothing but the training runner, and report fall rates that differ by a factor of three: in practice the culprit is rarely the algorithm and is usually the wrapper that quietly repacked the observations. Whether that policy walks on real hardware depends on the PhysX contact model (NVIDIA's GPU-accelerated rigid-body physics engine, which computes how the simulated robot's feet collide with terrain), the Proximal Policy Optimization (PPO) rollout semantics, the velocity-tracking reward terms, and the terrain-reset distribution all being versioned in one training artifact. Change the runner without pinning those, and the 256-seed holdout fall rate moves for reasons no one can later reconstruct.

This section develops the contract between an Isaac Lab environment and a training runner. The contract is concrete: observation groups, action clipping, reward terms, reset buffers, device, runner config, and evaluation script. As Figure 17.3A shows, one Isaac Lab task feeds several runner-specific training desks, and as Figure 17.3B details next, the wrapper is what adapts the same environment to each learner you choose.

One recurring pattern in that contract is worth naming up front: an asymmetric actor-critic setup, where the critic (the network that estimates how good a state is) sees more information during training than the actor (the network that produces actions) is allowed to see at deployment. The Theory section below defines exactly how observation groups implement that split; for now, treat the term as shorthand for "the critic gets privileged simulator state, the actor does not."

The key question is practical: if you switch from RSL-RL to rl_games or SKRL, which parts of the experiment are allowed to change, and which parts must remain identical for the comparison to mean anything?

Theory

Isaac Lab tasks commonly expose named observation groups. A policy actor may receive proprioception, commands, and history, while the critic may receive privileged simulator state such as terrain height, contact flags, or object poses. In asymmetric actor-critic training, those groups must be routed explicitly so privileged information helps value learning without leaking into the deployed actor.

Named observation groups matter for embodied AI because real robots have no simulator oracle. A legged robot cannot query terrain height ahead of its foot; a manipulator cannot read the internal pose of an object it has not yet touched. If training conflates simulator-only state with sensor-available state, the policy learns a dependency that cannot be satisfied on hardware, and the robot fails at the first contact with reality rather than in simulation where failures are free.

How the groups flow through the wrapper

Isaac Lab defines observation groups in the task configuration as named dictionaries, and each key maps to an observation term function. At each step, the simulator evaluates every term, concatenates the resulting tensors per group, and stores each group in a named GPU buffer. The wrapper then reads each buffer by name and routes it to the correct runner input slot. The actor receives one concatenated tensor; the critic receives a larger one that includes privileged keys. This data pipeline enforces the separation rather than relying on convention alone. A misconfigured wrapper that reads the wrong key either raises an error or silently widens the actor tensor, rather than corrupting reward computation directly. That last point is the counterintuitive part: reward curves can look perfectly healthy for 1500 PPO iterations while the actor has been secretly reading 52 terrain-height values it will never have on the real robot, so the policy that scores highest in simulation is precisely the one most likely to fall at the first outdoor test.

A policy trained on privileged simulator state it cannot access at deployment is a policy trained on a problem that does not exist in the real world. Beyond guarding that observation contract, the same wrapper layer determines how efficiently the rollout data moves, so it shapes throughput just as decisively as it shapes correctness. The runner boundary also controls performance. rl_games can work directly with GPU buffers, RSL-RL expects its own rollout storage conventions, and SKRL emphasizes transparent algorithm configuration across backends. The task is the scientific object; the runner is the optimizer and storage implementation. That distinction is what makes the wrapper the experiment boundary, not mere scaffolding.

Asymmetric Actor-Critic in Practice

Consider a concrete Anymal-D locomotion policy trained with RSL-RL on Isaac Lab's rough-terrain task using 4096 parallel environments. The critic receives 187-dimensional input including 52 terrain-height scan points; the actor receives 48-dimensional proprioception plus command velocity. After 1500 PPO iterations (roughly 20 minutes on a single A100, as Rudin et al. (2022) report), the critic estimates value accurately because it can "see" upcoming steps. The actor learns to navigate rough terrain from that informative value signal alone. Running the same PPO update loop on a single simulated environment typically reaches equivalent policy quality only after roughly 10 days of wall-clock time, by Rudin et al.'s own comparison against earlier CPU-bound pipelines. The massively parallel rollout recipe running 4096 GPU environments compresses that to 20 minutes, an approximately 700x speedup (on 2022 hardware) that is typically invisible to the algorithm and owned almost entirely by the wrapper layer.

At deployment on the real Anymal, the actor input remains 48-dimensional and the policy transfers without modification. The failure case is when a wrapper silently concatenates terrain heights into the actor tensor: the policy trains to use them, and the 52 terrain dimensions are simply absent on the real robot.

Worked Example

With the algorithm's contract specified in the abstract, the next step is to pin down exactly which fields that contract names, because those fields are what a runner swap is allowed or forbidden to touch.

Code Fragment 17.3.1 makes the runner choice explicit. It is a small manifest, but it captures the fields that are often hidden in launcher commands and YAML files.

# Compare Isaac Lab runner wrappers by the contract they must preserve.
# The task stays fixed while storage, observation packing, and logging vary.
from dataclasses import dataclass, asdict

@dataclass
class RunnerContract:
    runner: str
    wrapper_role: str
    actor_input: str
    critic_input: str

    def as_row(self) -> dict[str, object]:
        return asdict(self)

contracts = [
    RunnerContract("RSL-RL", "rollout storage for locomotion PPO", "obs", "privileged_obs"),
    RunnerContract("rl_games", "GPU buffer bridge and clipping", "obs", "states"),
    RunnerContract("SKRL", "readable algorithm and memory config", "states", "state_values"),
]

for contract in contracts:
    print(f"{contract.runner}: actor={contract.actor_input}, critic={contract.critic_input}")

Expected output: the trace should reveal which tensor group reaches the actor and which reaches the critic. If the manifest cannot answer that question, an asymmetric training result is not reproducible.

Practical Recipe

1. Define the Isaac Lab task first: robot asset, scene, observations, actions, rewards, terminations, curriculum, and randomization.
2. Choose the runner based on the experiment goal: speed, readable algorithm research, recurrent policies, or compatibility with existing locomotion configs.
3. Record actor observation groups and critic-only privileged groups before training.
4. Keep train and play scripts separate so evaluation uses deterministic actions and held-out seeds.
5. Export the checkpoint, normalization statistics, runner config, task config, and exact Isaac Lab commit or release together.

The idea in this section becomes useful when the runner boundary is explicit. Isaac Lab gives you a task graph; the runner gives you a learner. The wrapper is where the two meet, so it must be part of the experiment record.

The graduate-level habit is to separate task validity from runner performance. A runner can update faster without improving the task definition, and a better task curriculum can improve every runner. A fair comparison changes one of those layers at a time.

A robust implementation starts with a manifest that binds task, wrapper, runner, and evaluation. The manifest is small enough to review in a pull request and concrete enough to reproduce a run later.

1. Record the task entry point and runner wrapper in the same config artifact.
2. List actor observations and critic-only observations separately.
3. Store action clipping, observation clipping, and normalization settings.
4. Save train and play commands with explicit seed panels.
5. Compare runners only through the same play script and metric exporter.

# Record the Isaac Lab task, wrapper, and evaluation contract together.
# This prevents runner comparisons from hiding observation or seed changes.
from dataclasses import dataclass, asdict

@dataclass
class IsaacLabRunManifest:
    task: str
    runner: str
    wrapper: str
    actor_obs: tuple[str, ...]
    critic_obs: tuple[str, ...]
    eval_panel: str

    def as_row(self) -> dict[str, object]:
        return asdict(self)

manifest = IsaacLabRunManifest(
    task="Isaac-Velocity-Rough-Anymal-D-v0",
    runner="rsl_rl",
    wrapper="RslRlVecEnvWrapper",
    actor_obs=("proprioception", "commands", "history"),
    critic_obs=("terrain_heights", "contact_flags", "base_velocity"),
    eval_panel="rough_terrain_holdout_256",
)
print(manifest.as_row())

When an Isaac Lab run fails, inspect the wrapper contract before changing rewards. Common faults include critic-only state leaking into actor inputs, train-time randomization missing from evaluation, action clipping differing across runners, and normalization files not loaded during play.