Section 50.6: Ethical concerns

A robot cannot apologize its way out of a data-retention policy.

A Data-Retention Policy

A care robot's microphone runs through the night, its depth camera watches a resident dress, and its odometry quietly maps a daily routine: in embodied AI the question is never whether sensitive data is collected, but whether anyone can see it, consent to it, or stop it. Ethics in human-robot interaction (HRI), the study of how people and robots share space, tasks, and authority, becomes useful only when it is tied to a named interface, a replayable scenario, a failure diagnostic, and an artifact that records what changed in the action loop.

The key question is practical: Who is affected, what data is collected, what action can harm them, and who can inspect or stop the system?

Figure 50.6A makes the core design demand visible: the same system that acts in public must also expose who can consent, intervene, audit, and shut it down. Figure 50.6B renders this as a control loop, showing how consent, robot action, audit, harm check, and stop authority connect and feed back into one another.

Theory

The practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

Worked Example

Consider a care robot that records speech and movement to provide assistance. The system needs consent boundaries, data minimization, bias checks, transparent fallbacks, and human escalation.

Data minimization matters in embodied AI because robots operate continuously in intimate spaces. A care robot's microphone runs during sleep. Its depth camera captures body posture during dressing. Its odometry logs the resident's daily routine. Every retained byte creates a future attack surface, risks a regulatory violation, and preserves an irreversible record of a person's most private moments. A web service can pause collection; a robot cannot stop sensing without abandoning its task. Consider the scale. A single Intel RealSense D435i running at 30 fps for one 8-hour shift produces roughly 250 GB of raw depth frames before any compression. Every hour of inaction on a data-minimization policy adds another 30 GB of intimate behavioral record that no one can un-collect. Policy-layer restrictions are not enough. The system must enforce data limits at the hardware interface, before any network path exists.

Mechanically, data minimization works by applying four gates in order: resolution reduction (downsample before feature extraction), on-device feature extraction followed by raw frame deletion, time-bounded retention with automatic expiry, and purpose binding (tagging each retained snippet with the single task it was collected for, and rejecting any request that asks it to serve a different task) that blocks using a fall-detection audio clip for any query other than fall detection. Each gate reduces the exploitable surface while leaving the task signal intact.

Consider a concrete case. Stevie II, a socially assistive robot trialed in a care home (Trinity College Dublin, 2019), collected audio within a 1.5 m interaction radius. Designers set three explicit constraints: retention capped at 72 hours, on-device deletion of speaker-identification features before any cloud upload, and a physical mute button with an LED indicator visible from 3 m. When a resident did not press consent on the tablet prompt within 10 seconds, the robot defaulted to gesture-only mode and logged the timeout as an exclusion event. That log produced a measurable proxy: the exclusion rate across resident groups. In the first week, residents over 85 showed a 34% exclusion rate against 12% for residents aged 65 to 84, which flagged an accessibility gap in the tablet prompt font size. One configuration change fixed it, and the log artifact made the cause traceable. Without the logged exclusion metric, the disparity would have stayed invisible. This is the practical force of what designers call ethics as a measurement contract. Stated only in policy documents, the same principle went unaddressed for three prior deployments. Stated as a logged metric with a threshold, a team caught and fixed it in seven days.

# Simulate an HRI consent-and-exclusion audit: track per-group consent rates
# and flag accessibility gaps when exclusion exceeds a threshold

import numpy as np

rng = np.random.default_rng(42)

# Resident groups and their simulated consent-prompt response probabilities
groups = {
    "age_65_to_84": 0.88,   # 88% respond within timeout
    "age_85_plus":  0.66,   # 66% respond; font/accessibility gap
}

N_RESIDENTS = 200          # residents per group in the trial week
EXCLUSION_THRESHOLD = 0.20 # flag any group above 20% exclusion
TIMEOUT_SECONDS = 10       # seconds before robot defaults to gesture-only

audit_log = []

for group, p_respond in groups.items():
    # Bernoulli trial: did the resident respond in time?
    responded = rng.random(N_RESIDENTS) < p_respond
    n_consented = responded.sum()
    n_excluded  = N_RESIDENTS - n_consented
    exclusion_rate = n_excluded / N_RESIDENTS

    status = "OK" if exclusion_rate <= EXCLUSION_THRESHOLD else "FLAGGED"
    audit_log.append({
        "group": group,
        "consented": int(n_consented),
        "excluded":  int(n_excluded),
        "exclusion_rate": round(exclusion_rate, 3),
        "status": status,
    })

print(f"{'Group':<20} {'Consented':>10} {'Excluded':>9} {'Rate':>6}  Status")
print("-" * 58)
for row in audit_log:
    flag = "  <- accessibility gap detected" if row["status"] == "FLAGGED" else ""
    print(f"{row['group']:<20} {row['consented']:>10} {row['excluded']:>9} "
          f"{row['exclusion_rate']:>6.1%}  {row['status']}{flag}")

Group                Consented  Excluded   Rate  Status
----------------------------------------------------------
age_65_to_84               176        24  12.0%  OK
age_85_plus                132        68  34.0%  FLAGGED  <- accessibility gap detected

Practical Recipe

1. Define the sensor footprint before deployment: for a care robot using a depth camera (e.g., Intel RealSense D435i, 30 fps, 0.3 to 3 m range), specify which voxels require resident consent and which are discarded on-device before any upstream logging.
2. Set physical thresholds as first-class design parameters, not afterthoughts: maximum approach speed (e.g., 0.3 m/s for Stevie II in corridor mode), minimum stopping distance (0.5 m from a detected person), and force limits on any contact surface (the Franka Panda typically caps at around 30 N in collaborative mode via its joint torque sensors, though the exact limit is configurable per deployment).
3. Log every ethics-relevant event through the robot's existing telemetry pipeline rather than a separate system: in ROS 2, publish to /ethics_events with a stamped message so that consent timeouts, data-retention deletions, and fallback-mode activations appear alongside Inertial Measurement Unit (IMU) and odometry data in the same bag file and can be replayed for audit.
4. Record failures with the embodied context that caused them: not just "exclusion event" but "tablet prompt timed out after 10 s, resident age 87, ambient light 45 lux, robot pose 1.2 m at 210 degrees, font size 14 pt" so the root cause (lighting, distance, font) is separable.
5. Before any live human trial, run one sim-to-real perturbation in MuJoCo or IsaacSim: inject a 200 ms latency spike into the consent-response channel and confirm the fallback to gesture-only mode triggers within the specified timeout, because network jitter in hospital Wi-Fi regularly exceeds 150 ms.

Ethical concerns becomes useful when it is tied to a closed-loop contract for Human-Robot Interaction. The contract names the participants, observations, action authority, timing budget, logging artifact, and recovery rule. Without that contract, a system can look capable in a notebook while failing the first time a partner delays, a person corrects it, or a deployment scene changes.

Separate the conceptual claim, the systems claim, and the evidence claim. A plausible mechanism, a clean interface, and a closed-loop result are different claims; the section should keep their evidence separate.

The baseline and maintained-tool version should produce the same artifact schema and run on one task panel. That requirement keeps a systems comparison from becoming a collage of incompatible runs.

1. Write a one-paragraph task contract with observation, action, success, and failure fields.
2. Start with the smallest simulator, dataset, or wrapper that exposes the task contract faithfully.
3. Run one deterministic smoke test and one perturbation test before scaling.
4. Save a single result artifact containing configuration, seed, metrics, videos or traces, and failure labels.
5. Compare methods only when one script evaluates them on the same task panel.

When Ethical concerns fails, avoid labeling the whole method as weak. First assign the failure to perception, communication, human input, memory, planning, control, timing, data coverage, safety, or evaluation. Then rerun one controlled perturbation that isolates the suspected cause. This pattern turns a disappointing rollout into a reusable diagnostic asset.