"Diffusion and flow policies buy multimodal action generation at the price of sampler and timing discipline."
A Grounded AI Agent
A robot arm tasked with folding a towel must commit to a smooth, continuous trajectory dozens of times per second. Autoregressive token-by-token prediction is simply too slow and too jerky for that. Diffusion and flow matching change the contract: instead of predicting one discrete token, the model iteratively refines an entire action chunk from pure noise into a precise, coordinated motion. That shift, from sequence prediction to denoising, is why pi-zero and RDT-1B perform qualitatively better on dexterous manipulation than comparably-sized autoregressive VLAs. Here you will see how the denoising objective works, why flow matching further tightens latency, and how FAST tokenization lets a transformer recover much of that speed without sacrificing trajectory smoothness.
This section assumes familiarity with action chunking and denoising objectives from section 22.2 and section 22.4, and with flow matching from section 22.5. The action tokenization trade-off introduced here is examined in detail in section 34.5. Co-training strategies mentioned for pi-zero point five are developed further in section 34.6, and deployment latency constraints on diffusion samplers are treated in section 55.2.
Figure 34.4 should be read as a decoding contract: denoising or flow steps, conditioning tokens, action horizon, control rate, and safety filter must fit inside the robot loop.
The closed-loop decoding contract for diffusion and flow VLAs. Vision feeds the VLA core, whose action head must finish its denoising or flow steps fast enough that the action chunk reaches the controller before the control deadline. The feedback arrow is the key insight: failure evidence from the controller only re-enters the loop at the next decision, so any sampler budget that overruns the loop turns a reactive policy into a pre-recorded motion. This is the same diagram introduced as Figure 34.1.
Review and Consolidation
Curriculum, depth, and self-containment. RDT-1B, pi-zero, FAST, and pi-zero point five show that action generation is now the central VLA design axis. For Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five, the practical reading is to pin down the interface, assumptions, concrete example, and failure mode before comparing methods.
Production and evaluation contract. The durable comparison is action representation: diffusion, flow, or compressed autoregressive tokens. For Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five, treat the diagram, code, table, exercise, warning, and references as one evidence packet: boundary, artifact, tool choice, transfer check, failure mode, and source grounding.
Before accepting a Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five result, name the loop variable that changed, the tool that makes it reproducible, the failure that would fool the metric, and the source that backs the claim.
Write an evidence row for one diffusion or flow rollout: sampler steps, action horizon, control frequency, task metric, inference latency, and the failure label for late or unstable commands.
# Represent a short action chunk as a batch-ready tensor shape.
batch_size, horizon, action_dim = 32, 16, 7
def chunk_shape(batch_size: int, horizon: int, action_dim: int) -> dict[str, int]:
return {
"batch": batch_size,
"horizon": horizon,
"action_dim": action_dim,
"tokens_per_batch": batch_size * horizon,
}
print(chunk_shape(batch_size, horizon, action_dim))chunk_shape helper computes the tensor dimensions (batch, horizon, action_dim, tokens_per_batch) that an action-chunking policy must allocate before predicting a horizon of joint-space actions in one forward pass. Predicting a horizon of actions rather than one step matters because real robot joints have inertia and bandwidth limits: a single-step policy must re-query the model at every control tick, introducing jitter and inference stalls that physically destabilize delicate contacts. Grouping actions into a chunk lets the controller execute smooth motor commands even during the inference gap between queries. Mechanically, the action head receives the VLM context embedding once and outputs all H joint-space vectors simultaneously; the low-level controller then streams those H commands at the target frequency before the next chunk arrives.Use robomimic, Diffusion Policy, or LeRobot policy implementations to prototype action chunking before designing a new architecture. These libraries already manage temporal windows, normalization, batching, and rollout evaluation.
Diffusion and flow VLAs matter because some robot actions are better modeled as continuous trajectory distributions than as long symbol strings. These heads trade simpler decoding for richer motor expressivity on dexterous and high-rate tasks.
Why Continuous Action Heads Returned
Tokenizing action is attractive because it lets a VLA reuse language-model sequence machinery. The cost is that robot motion is continuous, high-frequency, and often multi-modal. A drawer can be pulled with slightly different wrist poses. A bimanual task can admit many coordinated trajectories. A single discrete next token can be too brittle for this geometry.
Diffusion and flow action heads address this by generating action chunks as continuous trajectories. Diffusion policies learn to denoise action sequences conditioned on observations. Flow matching learns a vector field that transports noise into actions. Both routes let the model represent multiple plausible futures without forcing every motor detail through a small set of bins.
RDT-1B and pi-zero are best read as VLA systems whose action head is a trajectory generator. The vision-language backbone supplies context, while diffusion or flow supplies smooth continuous control.
RDT-1B, pi-zero, pi-zero FAST, and pi-zero point five
Figure 34.4A shows the shared template: a VLM backbone encodes vision and language, and an action expert turns that context into action chunks (flow-matching for pi-zero, FAST tokenization for the autoregressive variant). Each system specializes the template. RDT-1B scales a diffusion transformer for bimanual manipulation, predicting chunks from language plus multi-view RGB. Pi-zero puts a flow-matching head on a pretrained VLM for continuous control across diverse robots. Pi-zero FAST returns to autoregressive generation with frequency-space token compression (FAST, where the action trajectory is transformed into the frequency domain and quantized so a standard next-token transformer can predict it without a diffusion or flow sampler). Pi-zero point five adds heterogeneous co-training to push open-world generalization on mobile manipulation.
These systems should not be collapsed into one category. RDT emphasizes bimanual diffusion at scale. Pi-zero emphasizes flow matching for general robot control. FAST emphasizes efficient action tokenization. Pi-zero point five emphasizes co-training across diverse sources for more robust generalization. What unites the flow-based members of this family is the denoising mechanism their action heads share, so it is worth building intuition for that mechanism before returning to the system-level differences.
Imagine sampling random action noise and learning a velocity field that moves it toward demonstrated action chunks. At inference time, the model follows that learned field for a small number of steps. The result is a continuous action sequence conditioned on the current scene and instruction.
The mathematical sketch is compact: learn \(v_\theta(x_t, t, c)\) so that samples move from a noise distribution toward demonstrated action chunks under context \(c\). The VLM backbone encodes that context, which includes images, language, and robot state. The practical question is how many flow or denoising steps the controller can afford before latency breaks the loop. For a 50 Hz controller, 10 flow steps at 5 ms each consume an entire control cycle before the first joint command is sent. Cutting to 2 steps reclaims 40 ms and keeps the loop fed. That margin is roughly the difference between a robot that recovers from a slipping grasp and one that drops the object every time.
Algorithm: Flow-Matching Action Generation (pi-zero)
Input: observation images \(o_t\), language instruction \(l\), robot state \(s_t\), number of flow steps \(K\), action horizon \(H\)
Output: action chunk \(\hat{a}_{t:t+H} \in \mathbb{R}^{H \times d_a}\)
- Encode \(o_t\) and \(l\) with the frozen VLM backbone to obtain context embedding \(c = \text{VLM}(o_t, l, s_t)\).
- Sample initial action noise \(x_0 \sim \mathcal{N}(0, I_{H \times d_a})\).
- Set step size \(\Delta t = 1/K\) and time index \(t_k = 0\).
- For \(k = 1, \ldots, K\): compute velocity \(v_\theta(x_{t_k}, t_k, c)\) using the flow-matching action expert \(\pi_\theta\).
- Update \(x_{t_{k+1}} = x_{t_k} + \Delta t \cdot v_\theta(x_{t_k}, t_k, c)\).
- Increment \(t_k \leftarrow t_k + \Delta t\).
- After \(K\) steps, set \(\hat{a}_{t:t+H} = x_1\) (the transported sample).
- Clip \(\hat{a}_{t:t+H}\) to the joint-space safety bounds \([\alpha_{\min}, \alpha_{\max}]\).
- Send action chunk to the low-level controller; execute the first \(\min(H, \text{replanning horizon})\) steps at the target control frequency.
- Repeat from step 1 on the next replanning cycle, using the updated \(o_t\) and \(s_t\).
Step-Through: Flow-Matching Action Generation
Trace the algorithm with \(K = 2\) flow steps, action horizon \(H = 1\), and a scalar action (\(d_a = 1\)) to keep the arithmetic visible. Step size \(\Delta t = 1/K = 0.5\). Suppose the VLM context \(c\) produces a velocity field that, near this region, returns \(v_\theta(x, t, c) = 0.8 - 0.3\,x\) (a learned field pulling samples toward roughly \(x = 2.67\)). Sample initial noise \(x_0 = -1.0\), time \(t_0 = 0\). Step 1: \(v = 0.8 - 0.3(-1.0) = 1.10\), so \(x_{0.5} = -1.0 + 0.5 \times 1.10 = -0.45\), and \(t \leftarrow 0.5\). Step 2: \(v = 0.8 - 0.3(-0.45) = 0.935\), so \(x_{1.0} = -0.45 + 0.5 \times 0.935 = 0.0175\). After \(K = 2\) steps the transported sample is \(\hat{a} = 0.0175\). Now clip to safety bounds \([\alpha_{\min}, \alpha_{\max}] = [-0.5, 0.5]\): \(0.0175\) is inside the range, so it passes unchanged and is sent to the controller. Notice that two coarse Euler steps land short of the field's fixed point at \(2.67\); this is exactly why diffusion heads with curved paths degrade at low step counts, while flow matching's straighter paths tolerate the truncation better.
Real-World Application: Laundry Folding
Physical Intelligence demonstrated pi-zero folding diverse, never-before-seen laundry items (shirts, shorts, towels) straight out of a dryer, a task that is fiercely multi-modal because crumpled cloth admits countless valid grasp-and-fold trajectories. The flow-matching action head generated smooth 50 Hz bimanual control that an autoregressive token policy typically could not match on this contact-rich, deformable-object task. This demo is frequently cited, though not universally, as one of the clearest public signals that continuous action heads had moved from benchmark curiosity toward a credible recipe for dexterous home manipulation, a claim that rests on this demonstration rather than on a controlled head-to-head benchmark.
When tuning the number of denoising or flow steps for a real-time controller, run a sweep over {1, 2, 5, 10, 20} steps and plot task success rate against per-step wall-clock on the target hardware before committing to a number. Pi-zero's published results used 10 flow steps; dropping to 2 steps on contact-rich tasks (cloth folding, insertion) typically costs 10-20 percentage points of success in practice (based on ablations reported in the pi-zero paper, Black et al., 2024), while dropping to 1 step often collapses to a mean-seeking trajectory (mean-seeking meaning the sampler averages across the multiple valid action modes instead of committing to one, producing a blended motion that matches none of them) that fails multi-modal tasks entirely. Profile with torch.cuda.Event timing rather than Python time.time() to avoid host-device sync noise that can make 2-step look artificially slower than 10-step. Set the step count in the policy config, not as a hardcoded constant, so it can be overridden per-robot without retraining.
Use diffusion or flow heads when the task needs smooth multi-step motor behavior, multiple plausible action modes, or dexterous contact. Use tokenized autoregression when discrete sequence modeling, fast sampling, or language-model compatibility dominates. Revisit the choice after measuring latency and closed-loop recovery, not before.
Consider a specific case: pi-zero controlling a bimanual folding task. The VLM backbone encodes a wrist-camera image and the instruction "fold the shirt in half." The flow-matching head starts from Gaussian noise in 7-dimensional action space and runs 10 denoising steps. Each step takes roughly 5 ms on an onboard GPU, so the head produces a 16-step action chunk covering 0.32 seconds at 50 Hz. The robot executes those 16 steps while it generates the next chunk in parallel. If the cloth shifts unexpectedly mid-chunk, the robot cannot correct until the next chunk arrives, so the effective reaction latency climbs to 320 ms rather than the nominal 20 ms per step. This is called the chunk commitment trap, and it is where diffusion and flow systems most often fail on reactive contact tasks. A policy that cannot update mid-chunk is not a reactive controller: it is a pre-recorded motion with a camera attached.
Think of a bus that leaves the depot on a fixed schedule and follows a pre-planned route: once it pulls away, the driver cannot make unscheduled stops even if a passenger shouts that they forgot their bag at the kerb. The chunk commitment trap works the same way. The robot commits to a pre-computed sequence of joint commands at the moment the chunk is sent, and no new sensory evidence can alter those commands until the bus completes its route and returns to the depot for the next plan. Shortening the route (smaller horizon) lets the bus turn around sooner; running two buses in alternating shifts (parallel generation) keeps the kerb covered without waiting.
A common assumption is that diffusion heads and flow-matching heads are interchangeable because both refine noise into actions iteratively. They are not. Diffusion policies follow curved probability paths (DDPM and DDIM, where the sampler removes noise according to a fixed variance schedule over many steps). Those curved paths degrade sharply when the step count drops below roughly five. Flow-matching learns straight-line probability paths. Those straight paths can be distilled to one or two steps without catastrophic quality loss. Treating the two as equivalent miscalibrates the latency budget. A practitioner who cuts a diffusion sampler to two steps expecting pi-zero-like behavior will get mean-seeking, contact-breaking motion instead. The action head architecture determines how aggressively you can compress the step budget. Evaluate that choice empirically on the target robot and control frequency before deployment.
A beautiful action distribution is not useful if inference misses the control deadline. Always report action horizon, inference time, control frequency, and whether the controller can reuse an action chunk while the next chunk is generated.
Diffusion and flow heads can fail in at least three distinct ways: (1) the sampler budget is cut too aggressively to meet the control deadline, degrading action quality below the threshold for contact-rich tasks; (2) the training distribution does not cover the current scene, causing the flow model to converge to a plausible-looking but incorrect trajectory that the robot executes confidently; (3) action chunking introduces a "commitment horizon" where the robot follows a stale plan for the full chunk duration even after a disturbance, a problem that worsens as horizon length grows. None of these failures produce obvious error signals at inference time without explicit monitoring of task-completion metrics.
Treat diffusion and flow vlas: rdt-1b, pi-zero, pi-zero fast, pi-zero point five like a control-room label. If the label does not tell a future debugger what moved, what sensed, or what failed, it is decoration rather than engineering knowledge.
Three active directions are reshaping diffusion and flow VLAs as of 2024-2026. First, consistency distillation for robot policies: adapting consistency models (Song et al., 2023) to action generation so that a full diffusion trajectory is compressed into one or two network evaluations without catastrophic quality loss on contact-rich tasks. Physical Intelligence's work on pi-zero (2024) showed the appeal of flow matching for speed, and several 2025 follow-ups (including work from CMU and Stanford) are pushing toward single-step deployment on 200 Hz impedance controllers. Second, hybrid discrete-continuous action heads: systems such as GR00T N1 (Bjorck et al., 2025, NVIDIA) combine a slow language-reasoning stream with a fast continuous-action stream, decoupling semantic planning from motor generation. The open question is how to train the gating mechanism that decides when to hand off between streams without supervised switch labels. Third, co-training diffusion heads on video prediction: using large internet video as an implicit action prior, where the denoising objective is defined over future image frames rather than joint angles, then projecting back to motor commands. Pi-zero point five (Physical Intelligence, 2025) uses heterogeneous co-training as a first step; the stronger version would unify video and proprioception in a shared latent space. An open problem suitable for a PhD thesis: current flow-matching action heads assume a fixed action horizon and control frequency at training time, making adaptation to a new robot or task frequency require full retraining. Designing a horizon-agnostic flow head that can be queried at arbitrary replanning rates without retraining is unsolved and directly limits cross-embodiment transfer.
Expected output: Diffusion and flow VLAs: RDT-1B, pi-zero, pi-zero FAST, pi-zero point five should leave a reproducible VLA evidence trace with checkpoint, action representation, robot interface, metric, and failure label.
Why might a bimanual manipulation task benefit from a diffusion or flow action head? Your answer should mention multi-modality, action chunks, and latency.
Diffusion and flow heads solve a physical problem that tokenized autoregression cannot: generating smooth, multi-modal trajectories for high-rate dexterous tasks such as bimanual cloth folding or peg insertion, where a discrete next-token prediction produces jerky, mean-seeking motion that breaks contact. The cost is a sampler budget: pi-zero uses 10 flow steps at roughly 5 ms each on an onboard GPU, giving a practical floor of about 50 ms of inference overhead before the first joint command is sent. On a 50 Hz controller, that overhead is one missed control cycle. On a 200 Hz impedance controller (a controller that regulates force and compliance at the contact point, rather than position alone) for a Franka Panda (a widely used 7-degree-of-freedom research robot arm) in contact-rich insertion, it is ten missed cycles, which is why deployment on fast manipulators requires either distillation to 1-2 flow steps or a parallel-generation pipeline that keeps the joint loop fed without stalling.
For each task, choose tokenized autoregression, diffusion, or flow: pushing a block, folding cloth, opening a drawer, and mobile pick-and-place. Give one reason and one evaluation metric for each choice.
Lab: Step Count vs. Multi-Modality
Goal: see empirically why low flow/diffusion step counts collapse multi-modal action distributions, the failure mode behind the chunk commitment and mean-seeking warnings above. Tools: Python, PyTorch, and the open-source Diffusion Policy or LeRobot repo (a CPU is sufficient; no robot needed). Setup: build a toy 2D dataset where the demonstrated action is bimodal, for example a target drawn with equal probability from \((+1, 0)\) or \((-1, 0)\) given the same observation, plus Gaussian noise. Train a small flow-matching head (a 3-layer MLP velocity field) and, for contrast, a DDPM diffusion head on the same data. What to vary: the number of sampling steps \(K \in \{1, 2, 5, 10, 20\}\) for each head, and the action-chunk horizon \(H \in \{1, 8, 16\}\). What to observe: sample 2000 actions per setting and plot a histogram. At \(K = 1\) you should see both heads collapse toward the mean \((0, 0)\), an action that matches neither mode and would break contact on a real task; as \(K\) grows the flow head recovers the two distinct modes faster than the diffusion head. Record the smallest \(K\) at which each head reproduces both peaks, then relate that threshold back to the per-step latency budget of a 50 Hz controller. Expected insight: straight-path flow matching tolerates aggressive step truncation while curved-path diffusion does not, which is exactly the latency-vs-quality trade that governs real deployment.
Project Ideas
Beginner (weekend): Implement a minimal diffusion policy for a 2D reaching task in PyBullet or Gymnasium: train a small DDPM-style denoising network to predict 8-step action chunks from a goal image, then sweep the number of denoising steps from 1 to 20 and plot success rate against inference latency. The key challenge is understanding how sampler step count interacts with action quality before you ever touch a real robot. Intermediate (1-2 weeks): Adapt the LeRobot flow-matching policy to a bimanual pick-and-place task in MuJoCo or Isaac Lab, comparing a 10-step flow head against a FAST-tokenized autoregressive head on the same demonstration dataset. The key challenge is wiring both action heads to the same VLM context embedding so the comparison isolates the action representation rather than the backbone.
What's Next?
Section 34.5 zooms in on action representation, including the FAST tokenizer.
FAST uses frequency-space compression to tokenize continuous action sequences for autoregressive VLAs. It is the key source for the chapter distinction between naive per-dimension binning and compressed action-sequence tokenization.
Pi-zero point five extends pi-zero through heterogeneous co-training for broader open-world generalization. It is useful for readers studying the frontier between task-specific robot policies and household-scale generalist behavior.
Bjorck et al. (2025). "GR00T N1: An Open Foundation Model for Generalist Humanoid Robots." arXiv.
GR00T N1 frames humanoid control as a dual-system VLA architecture with reasoning and fast action generation. It prepares the transition from Chapter 34 into Chapter 35 and the later humanoid chapter.
Liu et al. (2024). "RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation." arXiv.
RDT-1B studies diffusion transformers for language-conditioned bimanual manipulation at large scale. It is especially relevant for readers comparing tokenized autoregression with continuous denoising heads.
pi-zero uses a flow-matching action head on top of a pretrained vision-language backbone. The paper is central for understanding why continuous action generation became a serious alternative to discretized action tokens.
Chi et al. (2023). "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." arXiv.
Diffusion Policy established denoising over action sequences as a strong imitation-learning recipe. It gives the mathematical and practical background for diffusion heads in later VLA systems.