Section 33.8: Safe LLM-agent interfaces

"The danger is not that the model says something harmful. The danger is that it says something plausible, and the interface asks no further questions before sending it to the motors."

An Actuation Boundary Inspector

A single fluent sentence, "clear the aisle fast," can put a human in a hospital if nothing between the language model and the motors ever pauses to ask whether the words are safe to obey; the figure above is that missing pause drawn as an engineering envelope. The LLM should never be the final authority on actuation; typed interfaces, guards, monitors, and human escalation define what commands can reach the robot.

The closed loop below (Figure 33.8) names the four stations a proposal passes through: the instruction enters, the planner turns it into an action object, the tool API exposes that object for verification, and the verifier's evidence feeds back into the next decision.

Embodied systems interpose safety logic between LLM proposals and robot action so that language never becomes direct authority over hazardous motion.

The practical question is which safety properties can be checked automatically at interface time and which require escalation or hard-coded limits in the controller.

Theory

Let the LLM propose action object \(u_t\), and let a safety filter \(\sigma\) map that proposal and state estimate to an allowed action: $$a_t = \sigma(u_t, \hat s_t), \qquad \sigma : \mathcal U \times \mathcal S \to \mathcal A \cup \{\text{block}, \text{escalate}\}.$$ The filter may pass, modify, block, or escalate the action depending on geometric, task, or policy constraints.

This formulation matters because it places safety at the interface boundary, where the planner is still symbolic and the controller still has time to refuse. Once an unsafe instruction becomes continuous motion, the system has fewer and more expensive options. Ablation studies in agent-safety benchmarks (as of 2024) show the gap directly. Enforcing safety purely through prompt instructions cuts unsafe command pass-through by roughly 40%. Adding a two-field typed schema guard, such as requiring human_clear: bool and zone_id: str, cuts it by over 95%, because the schema check fires unconditionally regardless of what the model was told.

Worked Example

Code Fragment 1 applies a tiny safety shield to a proposed action. The shield can return `block` or `escalate` rather than pretending every proposal must map to some executable motion.

# Block high-risk actions that require human approval.
# A safety shield sits between symbolic planning and execution.
# The planner may propose; the shield may refuse.
proposal = {"action": "pick(glass)", "risk": 0.81}
approval_required = proposal["risk"] > 0.7
decision = "escalate" if approval_required else "execute"

print({"proposal": proposal["action"], "decision": decision})

The expected output is a semantically plausible proposal that the safety interface refuses to execute directly. The important detail is the presence of `decision='escalate'`, because safe embodied interfaces must treat blocking and human review as first-class outcomes rather than as logging side effects.

Practical Recipe

1. Define a typed proposal object whose fields are visible to the safety layer. A two-field schema requiring human_clear: bool and zone_id: str catches over 95% of unsafe proposals unconditionally, while the same rules written only in the prompt catch roughly 40%, because a schema check fires even when the model ignores every instruction it was given.
2. Check permissions, geometry, resource limits, and human-approval rules before execution.
3. Allow the shield to modify, block, or escalate, not only pass or fail.
4. Log blocked actions because they are evidence of what the planner tends to propose unsafely.
5. Keep low-level controller safeguards active even when high-level interface shielding is strong.

Being able to name the blocking rule, as the self-check demands, is possible only because the interface sits precisely on the seam between reasoning and motion. Safety interfaces are where symbolic AI and control engineering meet most directly. The LLM's proposal is high level and semantically rich; the shield translates that richness into admissibility checks over geometry, resources, and policy. This is one reason typed action objects expose the fields the shield actually needs, making safety a structural property of the interface rather than a hope expressed in prose.

A second lesson is that safety is layered. Interface shields catch semantic and policy-level mistakes early, while low-level controllers catch timing, force, and dynamics violations later. Neither layer can safely replace the other.

Whichever tools from the table you assemble into a shield, the layer only earns trust if it leaves a durable record of what it refused. Code Fragment 2 stores the blocked proposal and the rule that blocked it. This is the right artifact for improving both the shield and the planner, because it preserves what the model wanted to do and why the system refused.

Why blocked proposals must be logged

Evidence logging matters in embodied AI because a physical robot cannot replay a near-miss from memory. If you do not record a blocked action, post-incident analysis cannot see it, so the next deployment runs the same risk. In regulated environments, a safety case requires proof that the shield fired correctly under the actual operating conditions, not under a simulator, and the log supplies that proof.

The mechanism is straightforward: after every shield decision, the filter appends a fixed-schema tuple containing the raw proposal, the state snapshot at decision time, the computed risk score, the decision outcome, and the name of the rule that fired. Storing the raw proposal alongside the decision is essential because it separates planner improvement (reduce bad proposals) from shield tuning (adjust thresholds), preventing the common mistake of weakening safety rules to reduce false positives.

# Record a blocked proposal with the rule and state that caused the block.
# Keeping the raw proposal separates planner tuning from shield tuning.
record = {
    "proposal": {"action": "move_to(shelf=7)", "speed": 1.2},
    "state": {"human_clear": False, "zone_id": "cell-3"},
    "risk": 0.81,
    "decision": "block",
    "rule_fired": "human_clear_required_above_0.5",
}

print({"decision": record["decision"], "rule": record["rule_fired"]})

1. Log proposed actions before the shield rewrites or blocks them.
2. Store the specific safety rule and state evidence that fired.
3. Differentiate automatic replanning from human escalation in the planner state.
4. Audit blocked-action frequency by class to see where the planner needs stronger guidance.
5. Keep the same shield active during evaluation and deployment so safety metrics remain meaningful.

The expected output is a safety record that preserves the blocked action, the active rule, and the state evidence that triggered it. If future tuning reduced unnecessary escalations, this same record structure would show whether the gain came from better perception, better risk estimation, or a weaker shield.

If safe interfaces fail, check whether the proposal schema hid a crucial field, whether the wrong state variable drove the shield, or whether escalation policies were too weak for the task class. Safety bugs usually live at these boundaries, not in generic model capability.