Section 8.5: Sensor noise and uncertainty models

"Random noise you can average away. Structured noise will follow you into the state estimate and pretend to be truth."

Section 8.5

A LiDAR that reports 4.00 m to a glass door it cannot actually see, and an IMU that swears the robot is level while propeller vibration shakes its readings by a factor of five: both numbers look perfectly valid, and both will quietly poison a state estimate unless you can say, in advance, exactly how much to distrust them. This section builds the mathematical vocabulary for that gap between what a sensor reports and what is physically true. Every noise model here feeds directly into the Kalman and extended Kalman filters of section 8.6, where the \(R\) matrix you derive from real data determines whether the filter trusts an IMU reading during a vibration event or a LiDAR return near a glass surface. The three types of model covered are: additive Gaussian noise (the tractable baseline), multiplicative and context-dependent errors (bias, scale, and drift that change with temperature or contact), and fat-tailed outlier distributions (the non-Gaussian events that corrupt state estimates and cause falls or collisions if left ungated). By the end of this section you should be able to take a raw sensor log, fit a covariance to it, and check whether that covariance is calibrated (whether the claimed confidence region actually contains the true state as often as it says it does), not just recite the names of the three noise categories. Figure 8.5B traces how these three noise categories flow through the measurement model into the covariance matrix R that the downstream state estimator consumes.

Each model answers one practical question: given this sensor reading arriving at this timestamp, how much should the state estimator move its belief? A wheel encoder on a Franka Panda arm reports joint position to 0.1 mrad resolution under static load. Under dynamic load, flex in the link structure introduces an effective error of 0.3 to 0.8 mrad at the end-effector. A depth camera such as the Intel RealSense D435 has a depth noise standard deviation of roughly 2 mm at 0.5 m. That figure grows to 30 mm at 3 m and becomes bimodal near edges (a pixel straddling an object boundary partially images the near surface and partially the far surface, so its reported depth clusters around two separate values instead of one). The 30 mm figure already exceeds the 20 mm clearance margin a typical indoor doorframe navigation planner uses. Raw depth readings at 3 m therefore cannot safely guide a door traverse: the planner first needs a noise model that tells it to widen the safety margin. Knowing which model applies to which sensor at which operating point is what separates a filter that works in the lab from one that works on a real robot.

Theory

A noise model is only useful if you can inspect it before the filter starts trusting it. For a VLP-16 LiDAR on an autonomous forklift, that means recording in one artifact: the units (meters), the frame (sensor-to-base transform from the URDF), the per-beam range standard deviation measured against a flat wall (roughly 3 cm at 10 m for the VLP-16), the latency budget (the 100 ms it takes a full 600 RPM rotation to complete), and the failure labels (returns near the glass loading-dock door, where intensity collapses and range becomes meaningless). Skip any one of these and the downstream EKF in section 8.6 inherits a covariance it cannot defend.

Worked Example: Projecting a Point and Propagating Its Uncertainty

The right way to model a measurement is as a value plus a covariance. A 3D position estimate is a mean \(X_w\) with a covariance \(\Sigma_X\), and any function applied to that estimate transforms the covariance too. The pinhole camera is the canonical example. A world point projects to a pixel through the intrinsic matrix \(K\) and the extrinsic pose \([R\,|\,t]\):

where \(K\) holds the focal lengths and principal point, \([R\,|\,t]\) maps the world frame into the camera frame, and the scalar \(\lambda\) is the point depth that is divided out to land on the image plane. Calibration is the process of recovering \(K\) and the lens distortion from images of a known target. Because projection is nonlinear (the divide by depth), a 3D position uncertainty does not map to a simple fixed pixel uncertainty: depth error, in particular, blurs the pixel location more for nearby points. The example below projects one point and then propagates its 3D covariance to a pixel covariance by Monte Carlo, the same idea an extended Kalman filter would handle with a Jacobian. The code below makes that covariance-in, covariance-out contract concrete, and the fitting recipe that follows generalizes it.

# Project a 3D point to a pixel, then propagate its uncertainty to image space.
import numpy as np

K = np.array([[700.0,   0.0, 320.0],
              [  0.0, 700.0, 240.0],
              [  0.0,   0.0,   1.0]])
R = np.eye(3)                       # camera aligned with world axes
t = np.array([0.0, 0.0, 0.0])       # camera at the world origin
X_w = np.array([0.2, -0.1, 2.0])    # a 3D point, meters

Xc = R @ X_w + t                    # world -> camera frame
uvw = K @ Xc                        # apply intrinsics
u, v = uvw[0] / uvw[2], uvw[1] / uvw[2]   # divide by depth (lambda)
print(f"pixel = ({u:.1f}, {v:.1f})")

# 3D position uncertainty (depth noisier than lateral), propagated by sampling.
rng = np.random.default_rng(3)
Sigma_X = np.diag([0.01, 0.01, 0.04])     # variances in m^2
samples = rng.multivariate_normal(X_w, Sigma_X, size=5000)
proj = K @ samples.T
pix = (proj[:2] / proj[2]).T
print("pixel std (u, v):", np.round(pix.std(axis=0), 2))

Practical Recipe

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

Production Pattern

Sensor noise and uncertainty models sits inside the Part II robotics contract: geometry defines where things are, kinematics defines what motion is possible, dynamics defines what motion costs, control defines how errors are corrected, and sensing defines what the agent can know on time.

A noise model must state its assumed distribution, the conditions under which it breaks, and the test that catches overconfidence. That specification gives the idea an intuitive role, a formal interface, a runnable check, and a reproducible failure mode: enough for practitioners, builders, and researchers to use it at once.

Use this recipe when turning Sensor noise and uncertainty models into code, a simulator experiment, or a robot diagnostic. The point is not to use every library. The point is to keep the hand-built baseline and the maintained-tool path comparable.

1. Define each sensor message with units, frame, timestamp source, calibration file, and covariance meaning.
2. Run a static test, a slow-motion test, and a dropout test before fusing streams.
3. Compare the hand filter with FilterPy or ROS 2 robot_localization using identical measurements and noise settings.
4. Log innovation, covariance, delayed messages, rejected measurements, and downstream control effect.
5. Treat perception output as a belief with uncertainty, not as ground truth handed to the controller.

Uncertainty bugs masquerade as planning bugs when covariance is too small, non-Gaussian tails are ignored, or latency is treated as noise. Inspect residuals and confidence calibration before changing the controller. A concrete scale to hold in mind: in practice, teams that switch from a datasheet \(R\) to a measured \(R\) commonly report position-error reductions in the 40-60% range for the sensor and platform combinations described in this section, though the exact figure depends on how far the datasheet value was from operating conditions; the difference between those two setups is not a better algorithm but a single number set from real data.