Section 7.2: Feedback, error, stability, overshoot, oscillation

"Feedback is the thing that makes the system care about being wrong. It is also the thing that can make it care too much, too fast, and in the wrong direction."

A Control Theorist Watching an Actuator Shake

Watch a quadruped's ankle buzz at a steady frequency, or a drone's altitude jitter up and down by a few centimeters that never quite die out, and you are seeing the same culprit a control engineer sees in a step-response curve: a feedback loop fighting itself. A usable mental model turns that visible misbehavior into three moves: define the object of study, connect it to the agent loop, and test it with a compact implementation.

The key question is practical: what must the agent know, what can it observe, what action is available, and what evidence shows that the action worked under the stated conditions?

Theory

Consider a drone holding an altitude of 2 m. The onboard controller reads the barometer, computes an error (say, the drone is at 1.8 m, so \(e = 0.2\) m), and increases rotor thrust. If the gain is too low the drone climbs slowly and wind pushes it further down: the system is sluggish. If the gain is too high the drone overshoots to 2.4 m, the controller now commands a large downward correction, the drone drops to 1.6 m, and the cycle repeats: the system oscillates. The right gain produces a smooth, fast approach that settles at 2 m without ringing. To make this concrete: a proportional gain of 1.0 settles the drone in roughly 0.4 s with 5% overshoot; raising the gain to 10.0 produces 60% overshoot and the drone keeps ringing for over 8 s before settling. A 10x gain change turns a calm hover into a shaking chassis. Every concept in this section (error, stability, overshoot, oscillation, settling time) names one specific way this balance can go right or wrong. Figure 7.2A shows these outcomes side by side as step-response curves for underdamped, critically damped, and overdamped systems, with rise time, overshoot, and settling time marked on each.

From error signal to stability property

Before you can change any of those gains intelligently, you need to see the signals that the gain acts on, which is why the first practical move is to expose the interface rather than to tune.

The practical design rule is to make the interface inspectable before optimization begins: inputs, outputs, units, latency, bounds, and failure labels should all be visible in the saved artifact.

Feedback begins with an error signal, \(e_t=r_t-y_t\), where \(r_t\) is the reference and \(y_t\) is the measured output. Stability asks whether a small disturbance stays bounded or shrinks; overshoot asks how far the response passes the target; oscillation asks whether correction arrives with the wrong phase or too much gain. For an AI practitioner, these are not abstract labels. They are log columns in a step-response test. A controller with no error signal is not controlling anything: it is issuing commands into silence and hoping, the defining trait of open-loop control. Figure 7.2B traces how these quantities connect: the sensor feeds the measured output back to the summing junction, where it is subtracted from the reference to form the error that drives the controller.

The error signal matters in embodied AI because physical actuators cannot act on intent, only on numbers. A robot arm with no error signal has no information about whether its last command moved the end-effector toward or away from the target. Without that signed difference, every joint position command is open-loop: drift, gravity, and friction accumulate unchecked, and the arm wanders. In safety-critical settings such as surgical robots or legged locomotion over uneven terrain, even a 5 mm uncorrected positional error can cause tissue damage or a fall.

Mechanically, the error signal closes the loop by feeding back into the controller at every timestep. The controller multiplies \(e_t\) by one or more gains and sends the resulting torque or velocity command to the actuator. The plant responds, the sensor reads the new \(y_{t+1}\), and the controller computes a fresh error. This cycle repeats at the control rate (typically 100 Hz to 1 kHz on real hardware). To feel the stakes: for a typical mid-size robot arm, running at 100 Hz with no feedback accumulates roughly 2-3 cm of positional drift per second from gravity and joint friction alone; close the loop and that same arm can hold within 0.1 mm indefinitely, an illustrative 200-300x improvement from one subtraction (the exact ratio depends on the arm's mass, friction, and the tolerance band used to define "holding"). Every downstream property, stability, overshoot, oscillation, depends on how large and how quickly \(e_t\) changes across those cycles.

Before reading the table below, make a quick prediction: if you double the proportional gain on a system that is already showing 10% overshoot, does the settling time get shorter, longer, or does the system become unstable? Hold your answer, then check it against the step-response metrics that follow.

Reading this table in order gives the "which parameter to change first" rule promised at the start of this section: a slow rise time with no overshoot points to raising proportional gain; overshoot with ringing points to raising derivative gain or damping before touching proportional gain; a persistent steady-state error with a clean, non-oscillating response points to adding or increasing integral action; and an oscillation that will not settle at any gain points away from the gains entirely and toward delay, sample rate, or sensor noise as the root cause.

Worked Example

Stability of a linear system is decidable by inspection of one matrix. For \(\dot x = Ax\), the equilibrium is asymptotically stable exactly when every eigenvalue of \(A\) has a strictly negative real part; engineers call this the left-half-plane condition, and it is the single most useful stability test in practice. The Lyapunov view gives the same answer through an energy-like function: if there is a positive-definite \(P\) with \(\dot V(x) = x^\top(A^\top P + P A)x \le 0\), the system is stable. Code Fragment 7.2.1 takes a pendulum linearized near upright (an unstable plant), closes a PD (Proportional-Derivative) loop around it, and verifies stability both ways. The full PID controller that builds on this PD core is tuned against exactly these overshoot and settling-time targets in the next section.

import numpy as np
from scipy.linalg import solve_lyapunov

def classify(A):
    ev = np.linalg.eigvals(A)
    mx = np.max(ev.real)
    tag = "stable" if mx < 0 else ("marginal" if np.isclose(mx, 0) else "UNSTABLE")
    return ev, mx, tag

# Pendulum near upright: gravity term gives a positive eigenvalue -> it falls.
A_open = np.array([[0.0, 1.0],
                   [16.0, -0.2]])
# Proportional-derivative feedback u = -[k1 k2] x folded into the dynamics.
k1, k2 = 30.0, 8.0
A_cl = np.array([[0.0,        1.0],
                 [16.0 - k1, -0.2 - k2]])

for name, A in [("open-loop", A_open), ("PD closed-loop", A_cl)]:
    ev, mx, tag = classify(A)
    print(f"{name:>15}: eig = {np.round(ev, 3)}  max Re = {mx:+.3f}  -> {tag}")

# Lyapunov certificate for the stable loop: solve A^T P + P A = -Q, P must be PD.
Q = np.eye(2)
P = solve_lyapunov(A_cl.T, -Q)
print(f"Lyapunov P eigenvalues = {np.round(np.linalg.eigvalsh(P), 3)}  (all > 0 proves stability)")

Practical Recipe

1. Write the observation, action, and success metric before choosing a model.
2. Build a baseline that is simple enough to debug by inspection.
3. Add the library implementation only after the baseline behavior is understood.
4. Record failures as structured cases: perception error, state error, planning error, control error, or evaluation error.
5. Run at least one perturbation test before trusting the result.

Production Pattern

Feedback, error, stability, overshoot, oscillation sits inside the Part II robotics contract: geometry defines where things are, kinematics defines what motion is possible, dynamics defines what motion costs, control defines how errors are corrected, and sensing defines what the agent can know on time. These corrections ultimately serve the perception-action loop that gives embodied agents their defining character.

Translate feedback behavior into measurable overshoot, settling time, steady-state error, and oscillation. That translation serves practitioners and researchers alike: it gives the idea an intuitive role, a formal interface, a runnable check, and a reproducible failure mode.

Use this recipe when turning Feedback, error, stability, overshoot, oscillation into code, a simulator experiment, or a robot diagnostic. The point is not to use every library. The point is to keep the hand-built baseline and the maintained-tool path comparable.

1. Write the control objective, measured state, actuator command, update rate, and saturation policy.
2. Run a step-response test before adding learning, with overshoot, settling time, and steady-state error logged.
3. Compare the hand controller with python-control, CasADi, Drake, do-mpc, or ROS 2 control on the same plant model.
4. Record latency, missed deadlines, saturation events, constraint violations, and recovery actions.
5. Only compare controllers and policies when they share sensors, action limits, disturbance tests, and safety checks.

A learned policy can hide an unstable inner loop until the disturbance changes. Consider locomotion controllers trained in simulation: a policy trained on a flat-ground MuJoCo model, then deployed on Boston Dynamics Spot, often shows a sim-to-real gap as oscillation in the ankle or knee joints once contact forces exceed the training distribution. The reward climbed in simulation, yet a marginally stable joint resonance persisted the whole time. Check update rate, delay, saturation, integral windup, and fallback behavior before scaling training. For this section, first reproduce one tiny step response by hand, then rerun it through python-control, CasADi, Drake, do-mpc, or ROS 2 control. If the two disagree, inspect sample time, sign convention, measurement delay, and output units before changing the model.