Section 7.5: Model predictive control (MPC) as receding-horizon optimization

"MPC commits to the first step of a plan it will never finish, because the plan is not the point; the planning is."

A Receding-Horizon Optimist

Picture a controller that draws up a careful fifteen-step plan, then deletes fourteen of the fifteen steps and starts over: that deliberate amnesia, repeated dozens of times a second, is exactly what lets MPC drive a car within its lane, land a rocket, and keep a humanoid upright on ice. Figure 7.5A captures the whole loop at a glance: solve a short-horizon optimization, execute only the first action, re-solve from the new state, and repeat. This section defines the object, connects it to the agent loop, and tests it with a compact implementation.

The key question is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

The practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

That inspectable interface is not a teaching abstraction; it is exactly what production controllers expose, because the same fields decide whether a real robot meets its deadline. MPC runs in production across many timescales and constraint types. Boston Dynamics runs receding-horizon whole-body MPC on Spot and Atlas at roughly 500 Hz. The optimizer enforces joint torque limits and foot-contact force constraints on every tick. Tesla's Autopilot path tracker uses a kinematic MPC with a 2-second horizon to stay within lane boundaries. SpaceX's Falcon 9 landing controller solves a convex fuel-optimal MPC problem in real time during final descent. Every deployment shares the same structure: the optimizer enforces constraints, and the receding horizon corrects model error.

MPC solves a short planning problem at every control tick, executes only the first command, then replans after the next observation. A model with 5% parameter error can accumulate a trajectory deviation of 30-50% over a 2-second open-loop plan. The exact figure depends on system nonlinearity and error structure. The same model under a 200 ms receding horizon typically keeps deviation well below 10%, because each tick corrects before errors compound. This is the error-before-it-compounds principle. It explains why replanning frequency often matters more than model fidelity. A model that is only 95% accurate still controls a robot safely if you correct at 50 Hz. Run that same 95%-accurate model open-loop for 2 seconds and it produces errors larger than the robot can recover from. The receding horizon also handles constraints directly: actuator limits, collision margins, contact forces, battery limits, joint limits, and comfort bounds all enter the optimization explicitly. The cost encourages progress; the constraints bound what the robot may spend to achieve it. Figure 7.5B traces one turn of this loop: the optimizer plans a full horizon, executes a single action, observes the resulting state, and re-solves with the window shifted forward.

Worked Example

The code below optimizes over the control sequence alone and recovers the states by rolling the dynamics forward from them, a pattern named "single-shooting" when it is introduced formally in the Practical Recipe; keep the label in mind now so the later definition reads as a name for something already familiar rather than a new idea.

MPC solves a finite-horizon optimization at every tick, executes only the first command, then replans. Code Fragment 7.5.1 implements this on a 1D double integrator (state \([\text{pos},\text{vel}]\), control = acceleration). Each tick minimizes \(\sum_{k=0}^{N}\lVert x_k - x_d\rVert_Q^2 + \lVert u_k\rVert_R^2\) subject to the discrete dynamics and a hard acceleration limit \(\lvert u\rvert \le 1.5\). The previous solution is shifted forward as a warm start, where a warm start is an initial guess handed to the solver so it begins near the answer instead of from scratch, which is what keeps the per-tick solve fast enough to meet a control deadline.

import numpy as np
from scipy.optimize import minimize

# 1D double integrator: state [pos, vel], control = acceleration. Drive to x_d.
dt, N = 0.1, 15                          # horizon length
A = np.array([[1, dt], [0, 1.0]])
B = np.array([0.5 * dt * dt, dt])
x_d = np.array([1.0, 0.0])
Qx, Qv, Ru = 10.0, 1.0, 0.1
u_lim = 1.5                              # acceleration constraint

def rollout(u_seq, x0):
    x, xs = x0.copy(), [x0.copy()]
    for u in u_seq:
        x = A @ x + B * u; xs.append(x)
    return np.array(xs)

def cost(u_seq, x0):
    e = rollout(u_seq, x0) - x_d
    return Qx * np.sum(e[:, 0] ** 2) + Qv * np.sum(e[:, 1] ** 2) + Ru * np.sum(u_seq ** 2)

def mpc_step(x0, warm):
    res = minimize(cost, warm, args=(x0,), method="SLSQP",
                   bounds=[(-u_lim, u_lim)] * N, options={"maxiter": 50, "ftol": 1e-6})
    return res.x, res.success

x, warm, solves_ok = np.array([0.0, 0.0]), np.zeros(N), 0
for t in range(40):                                  # closed-loop receding horizon
    u_seq, ok = mpc_step(x, warm)
    solves_ok += int(ok)
    u0 = float(np.clip(u_seq[0], -u_lim, u_lim))     # execute only the first command
    x = A @ x + B * u0
    warm = np.r_[u_seq[1:], 0.0]                     # shift solution for the warm start
print(f"feasible solves: {solves_ok}/40")
print(f"final state pos={x[0]:.3f} vel={x[1]:+.3f} (target pos=1.000 vel=0.000)")

This is where the "optimization" in the section title actually lives: Qx = 10.0, Qv = 1.0, and Ru = 0.1 are not arbitrary, they set the relative price the optimizer pays for position error, velocity error, and control effort. Raising Qx relative to Ru makes the optimizer spend more acceleration to close the position gap faster, at the cost of a sharper, more actuator-hungry trajectory; raising Ru instead produces a gentler approach that may miss the target within the horizon. The practical tuning loop is: start with the actuator limit fixed, set Qx an order of magnitude above Qv so position error dominates velocity error, then increase Ru only if the resulting trajectory saturates the actuator more often than the application can tolerate, rerunning the closed loop after each change rather than reasoning about the weights in isolation.

Practical Recipe

The toy double integrator hid the one decision that dominates every real deployment: the order in which you fix the control rate, the horizon, and the constraints, because each choice silently bounds the next.

1. Fix the control rate first and derive the solve budget from it: a Franka Panda torque loop at 1 kHz gives you 1 ms per tick; Spot's whole-body MPC at 500 Hz gives 2 ms; a Tesla-style lane tracker at 50 Hz gives 20 ms. Every later choice (horizon length, model fidelity, solver) is bounded by this number.
2. Build the baseline as a single-shooting QP, where single-shooting means the only decision variables are the control inputs and the states are recovered by rolling the dynamics forward from them, on the linearized dynamics around the current operating point, then check it open-loop against a MuJoCo or Drake rollout of the true nonlinear plant before closing the loop.
3. Choose the horizon \(H\) so the prediction spans at least one dominant plant time constant (roughly 1.5-2 s for a quadruped gait cycle, 200-400 ms for a manipulator reach), then shorten it until the worst-case solve fits the budget from step 1.
4. Encode actuator and safety limits as hard constraints inside the QP (joint torque on the Panda, foot-contact friction cones on Spot (the friction cone is the set of contact forces a foot can exert without slipping, bounded by the friction coefficient), lane boundaries on the vehicle), never as post-hoc clipping, and define the fallback command issued on infeasibility or deadline miss.
5. Stress the closed loop with the disturbance the controller will actually face: a 5% mass error on the quadruped, a friction-cone violation on ice, or a 1-tick sensor delay, and confirm the receding horizon corrects it within a few ticks rather than diverging.

Production Pattern

Model predictive control (MPC) as receding-horizon optimization sits inside the Part II robotics contract: geometry defines where things are, kinematics defines what motion is possible, dynamics defines what motion costs, control defines how errors are corrected, and sensing defines what the agent can know on time.

For MPC, log horizon, constraints, solver status, warm start, and missed-deadline behavior. The idea has an intuitive role, a formal interface, a runnable check, and a failure mode that can be reproduced.

A controller that plans perfectly over ten seconds but never corrects is not a controller; it is a schedule, and schedules break the moment reality disagrees.

Use this recipe when turning Model predictive control (MPC) as receding-horizon optimization into code, a simulator experiment, or a robot diagnostic. The point is not to use every library. The point is to keep the hand-built baseline and the maintained-tool path comparable.

1. Write the control objective, measured state, actuator command, update rate, and saturation policy.
2. Run a step-response test before adding learning, with overshoot, settling time, and steady-state error logged.
3. Compare the hand controller with python-control, CasADi, Drake, do-mpc, or ROS 2 control on the same plant model.
4. Record latency, missed deadlines, saturation events, constraint violations, and recovery actions.
5. Only compare controllers and policies when they share sensors, action limits, disturbance tests, and safety checks.

A learned policy can mask an MPC timing failure until the disturbance changes. Before scaling training, audit horizon length, model mismatch, constraint scaling, solver status, missed deadlines, warm start, and fallback behavior. Reproduce one small receding-horizon case by hand, then rerun it through CasADi, do-mpc, or Drake. When the two disagree, inspect dynamics discretization, constraint units, terminal cost, and the command actually executed after replanning.