Section 10.5: Rendering, logging, and debugging

"Rendering shows you what happened. Logging shows you what the agent believed happened. Debugging is the argument between the two."

A Simulator Diagnostic Loop

In practice, "logging" here means writing each step record to a durable file, such as a CSV or a line-delimited JSON file, keyed by seed and step index, rather than printing to the console: the worked examples below build the in-memory row structure first, and that structure serializes directly to either format with one extra line (for example, csv.DictWriter or json.dumps per row).

What This Section Builds

A policy can run for ten million steps, post a rising reward curve, and still be quietly cheating the simulator the entire time, and the only way you will ever find out is the evidence trail you decided to keep before the run started. Rendering, logging, and debugging are that trail: rendering shows what the environment believes is happening, logging preserves what happened, and debugging connects those records to a concrete failure cause. Figure 10.5A shows this workflow in action: a render frame at a key timestep sits beside the logged reward curve and an episode-length histogram, and together they pinpoint where the agent gets stuck.

The goal is to stop treating a reward curve as the whole story. An embodied environment should produce enough trace evidence to answer which observation arrived, which action was sent, what the environment returned, and why the episode ended. In practice, this is called the render-log gap, and it is where most simulation bugs hide: in typical practice, a policy trained for millions of steps with only a reward scalar can take days to debug; the same failure identified from a short render-plus-log trace can take minutes.

The render-log gap matters in embodied AI because physical robots have no rewind button. A gap that goes undetected in simulation transfers directly to hardware. A grasping policy that exploits a contact bug never seen in logs will apply the same exploit on a real arm. That exploit causes drops, collisions, or joint limit violations that damage equipment. The gap grows most dangerous when the reward improves despite wrong behavior, because the learning signal suppresses the investigator's motivation to look deeper.

Why the two views diverge

The gap arises because rendering and logging read different views of the simulator state. The renderer queries the full physics state, including geometry, contact normals, and visual meshes, and produces a pixel or text snapshot. The observation returned to the policy filters that same state down to a lower-dimensional projection, and the environment author decides what the info dictionary exposes. When those projections omit a variable, the renderer still shows it visually while the log records nothing. Only a side-by-side comparison of both artifacts reveals the discrepancy.

Theory

Gymnasium environments declare render modes such as human, rgb_array, or ansi. The right mode depends on the artifact: a live window is useful for local debugging, an RGB array can be saved as video, and text rendering can be checked in automated tests.

Choosing a render mode fixes only the visual half of the trace; the other half is what you write down on every step. Logging should sit next to the environment loop rather than after training. Each step record should include step index, seed, action, reward, a termination flag (terminated, meaning the episode ended because of an event inside the task itself, such as a fall or a completed goal), a truncation flag (truncated, meaning the episode was cut off by an external limit such as a step count, not by the task logic), and selected info fields (the info dictionary is a free-form mapping of diagnostic values that the environment author chooses to expose alongside the observation). For robotics, add controller status, contact events, safety margins, and timing.

Worked Example

Code Fragment 10.5.1 uses an ansi render mode (a text-based render mode that returns a printable string instead of an image, so it works over a remote terminal or in automated tests with no graphics window) so the example works without a graphics window. The render frame gives a human-readable view, while the step return gives the machine-readable trace.

# Use text rendering when a debug check should run without a GUI.
# The step trace still records reward, ending flags, and info keys.
import gymnasium as gym

env = gym.make("FrozenLake-v1", render_mode="ansi", is_slippery=False)
observation, info = env.reset(seed=3)
frame = env.render()
visible = [line for line in frame.splitlines() if line.strip()]

observation, reward, terminated, truncated, info = env.step(1)
clean_row = visible[0].replace("\x1b[41m", "[").replace("\x1b[0m", "]")

print(clean_row)
print({"obs": int(observation), "reward": reward, "ended": terminated or truncated, "info_keys": sorted(info.keys())})
env.close()

The expected output combines a human-readable render frame with a machine-readable step record. The frame shows the current grid state, while the dictionary confirms that the sampled transition did not end the episode and that a probability diagnostic is available in info.

Practical Recipe

1. Choose a render mode that matches the artifact: live inspection, saved video, image array, or text trace.
2. Log one row per environment step with action, reward, ending flags, and selected info.
3. Save the wrapper stack and render mode with the log.
4. When a rollout fails, classify the failure before changing the policy.
5. Keep two representative failure traces for each reported metric table.

Rendering records what the environment displays; logging records what the algorithm saw and optimized. A strong debugging workflow keeps the two synchronized by seed and step index. A reward curve that rises while the render trace shows a colliding arm is not progress but a hidden simulator exploit.

The graduate-level habit is to require traceability from a reported number back to at least one representative episode, the same discipline that the evaluation protocol and seeding rules build on. A success rate without failure traces is fragile because it cannot show which assumptions survived contact with the simulator.

With those tool roles settled, the question becomes how to bind them together into a single inspectable record. A robust debugging implementation starts with a tiny trace format. The trace should be small enough to inspect by hand and structured enough to join with videos, metrics, and safety events.

1. Choose the minimal render mode that captures the failure evidence.
2. Write one log row per step before training long runs.
3. Include terminated, truncated, and selected info fields in each row.
4. Save seeds and wrapper stack beside the trace.
5. Review a few failure traces before tuning reward or model architecture.

# Record a compact step trace that can be inspected after rollout.
# Each row preserves reward, ending status, and diagnostic keys.
import gymnasium as gym

env = gym.make("CartPole-v1")
observation, info = env.reset(seed=17)
env.action_space.seed(17)
trace = []

for step_index in range(3):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)
    trace.append({
        "step": step_index + 1,
        "action": int(action),
        "reward": float(reward),
        "ended": terminated or truncated,
        "info_keys": sorted(info.keys()),
    })

print(trace)
env.close()

The expected output is a short rollout ledger with one dictionary per step. Read it as a minimal debugging artifact: every action, reward, and ending flag is preserved in order, so a later aggregate return can still be traced back to concrete behavior.

When an experiment about rendering, logging, and debugging fails, avoid labeling the whole method as weak. First assign the failure to perception, state estimation, planning, control, timing, data coverage, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.