Section 48.6: Scenario testing and safety cases

A passing benchmark says the car did well on the test you wrote; a safety case says you wrote the right tests, and what is left when they pass.

On safety assurance for autonomous driving

A self-driving stack can pass one hundred percent of its test suite and still be unsafe to deploy, and the reason is not a bug but a definition: safety here is a measurable contract, not a score. That contract has four moving parts: define the Operational Design Domain (ODD, the explicit envelope of conditions a system is built and validated for, detailed below), enumerate hazards within it, mitigate them, and quantify the residual risk with scenario evidence tied to closed-loop logs. The recurring discipline is that a benchmark number is evidence inside a safety case, never a substitute for one. Figure 48.6A captures this view: a safety case is an argument that traces each hazard through its cause and mitigation down to a residual risk that both a regulator and an engineer can inspect.

Theory

Operational Design Domain

The ODD is the explicit envelope of conditions the system is built and validated for: road types, speed limits, weather, lighting, geography, and traffic rules. As a concrete example, Waymo's robotaxi ODD has historically been structured roads in geofenced metro areas, below roughly 70 mph, in defined weather. A vehicle outside its ODD (an unmapped construction reroute, a snowstorm beyond the certified envelope) must detect the boundary and execute a minimal-risk maneuver rather than continue.

The ODD boundary matters because the vehicle's actuators, sensor models, and learned policies were validated only inside it. Cross the boundary and perception degrades in ways the system cannot self-diagnose: a radar tuned for clear highway returns misfires in wet snow, and a lane-keeping network trained on mapped roads fails on a coned-off detour. A wrong action at 60 mph cannot be undone, so the boundary triggers a safe stop rather than a best-effort continuation.

ODD monitoring fuses several runtime signals. The localization stack compares the current GPS and map tile against the certified geography. Weather classifiers read camera histograms and wiper-state sensors. Speed telemetry checks against the limit table. When any signal crosses its certified threshold, a dedicated boundary monitor sets a flag that overrides the planner and commands a pull-over trajectory. This monitor is a separate, simpler module so an auditor can verify its logic independently, which keeps the safety-critical boundary check outside the complex learned planner.

Here is a question worth sitting with before reading on: if every sensor is working, every component behaves as designed, and the vehicle still strikes a pedestrian, which engineering process failed? The answer is the subject of the next standard.

SOTIF: Safety of the Intended Functionality (ISO 21448)

Classic functional safety (ISO 26262) handles faults: a sensor breaks, a wire shorts. SOTIF handles the harder AV problem of hazards with no fault at all, where every component works as designed but the function is insufficient. Its categories include performance limitations (the perception system genuinely cannot resolve a pedestrian in glare) and foreseeable misuse (a driver over-trusts a driver-assist feature). SOTIF drives you to shrink the space of unknown-unsafe scenarios through analysis and testing.

Shrinking that space is only half the task; the argument that the remaining space is acceptable has to be written down in a form others can inspect, which is what the safety case structure provides.

Safety case structure

A safety case is a traceable chain: hazard (unintended braking on the highway) leads to cause (false-positive detection of a phantom obstacle) leads to mitigation (multi-frame confirmation plus radar cross-check before braking) leads to residual risk (the quantified, accepted remainder after mitigation). Every link must point to evidence: a scenario suite, closed-loop logs, and a metric, not merely a claim. This discipline is called hazard-driven coverage tracing, and it separates a safety case from a benchmark report; Figure 48.6B lays out the four-link chain and the evidence layer beneath it. One concrete illustration shows why it matters. The NHTSA analysis of the Uber Tempe crash identified 37 distinct scenario variants for a pedestrian-with-vehicle crossing at night. Uber's pre-deployment test suite covered fewer than 5 of them. The aggregate pass rate never surfaced the gap between 37 hazard variants and 5 exercised scenarios. A safety case is a structured argument, not a score: passing every test you wrote proves only that you wrote passing tests.

Consider the combinatorial scale. A modest ODD with 10 hazard types and 8 slices (road type, lighting, weather) needs at least 80 scenario authorings before coverage is even theoretically complete. Most pre-deployment suites contain fewer than 20. So the average team ships with under 25 percent ODD coverage while the aggregate pass rate reads 100 percent.

Worked Example

The example builds a tiny safety-case checker: it scores closed-loop scenario logs against a hazard list, flags scenarios whose minimum time-to-collision violates the safety threshold, and reports ODD coverage so an untested slice cannot pass silently.

# Each scenario log records its ODD slice, the min time-to-collision (s),
# and which hazard it was designed to exercise.
logs = [
    {"id": "S1", "odd": "urban_day_rain",  "min_ttc": 2.4, "hazard": "phantom_brake"},
    {"id": "S2", "odd": "urban_day_clear", "min_ttc": 0.8, "hazard": "cut_in"},
    {"id": "S3", "odd": "highway_day",     "min_ttc": 3.1, "hazard": "lead_brake"},
]
required_odd = {"urban_day_clear", "urban_day_rain", "highway_day", "urban_night"}
TTC_FLOOR = 1.5  # seconds; below this is a residual-risk hazard

# Evaluate each scenario against the safety threshold.
violations = [s for s in logs if s["min_ttc"] < TTC_FLOOR]
for s in logs:
    status = "FAIL" if s["min_ttc"] < TTC_FLOOR else "ok"
    print(f"{s['id']} [{s['odd']:>16}] hazard={s['hazard']:<13} "
          f"min_ttc={s['min_ttc']:.1f}s -> {status}")

# ODD coverage: which certified slices have NO evidence at all.
covered = {s["odd"] for s in logs}
uncovered = required_odd - covered
print("\nResidual-risk violations:", [s["id"] for s in violations])
print("Uncovered ODD slices (no evidence):", sorted(uncovered))
print("Safety case PASSES:", not violations and not uncovered)

Expected output: S2 fails (a cut-in with 0.8 s time-to-collision, below the 1.5 s floor), and urban_night is reported as an uncovered ODD slice. The safety case does not pass, correctly, because there is both a quantified violation and a slice of the certified domain with no evidence at all. This is the structural point: passing tests plus missing coverage is still a failing safety case.

Practical Recipe

1. Write the ODD explicitly: road types, speeds, weather, geography, traffic rules.
2. Enumerate hazards (HAZOP plus SOTIF analysis) and map each to a cause and a mitigation.
3. Author scenarios that exercise each hazard across the ODD parameter space, including the long tail.
4. Run closed-loop, log per-scenario safety metrics, and compute coverage against the hazard list and ODD.
5. Quantify residual risk for what remains untested or only probabilistically mitigated; record it in the safety case.