Section 11.5: The Newton Physics Engine and OpenUSD

"A new simulator is a research claim until your task says it is an engineering dependency."

A Skeptical Physical AI Agent

Why Newton Exists

Three rival labs that usually compete (NVIDIA, Google DeepMind, and Disney Research) agreed that no existing simulator could train their next humanoid, so in 2025 they built a new physics engine from scratch. As Figure 11.5A captures, such a frontier engine earns its place in a pipeline only once baselines, solver checks, and reproducible scenes make its claims testable. Three concrete bottlenecks drove Newton's creation. First, training a whole-body locomotion policy for a humanoid such as Agility Robotics' Digit requires rolling out tens of thousands of parallel environments; MuJoCo's CPU-threaded backend saturates a 64-core machine before reaching the parallelism that Proximal Policy Optimization (PPO) or Soft Actor-Critic (SAC) needs for sample-efficient convergence, whereas Warp-backed GPU kernels push that limit past 32,768 simultaneous environments on a single A100. Second, scene-composition pipelines for manipulation now live in USD. Disney's production assets, NVIDIA's Omniverse robot libraries, and Google DeepMind's tabletop scene generators all emit OpenUSD. A simulator that treats USD as a first-class scene graph rather than an import afterthought eliminates a hand-conversion step that can silently corrupt contact geometry. Third, MuJoCo's contact solver (a convex-complementarity formulation, meaning it models contact as a convex optimization problem where bodies cannot interpenetrate) and Isaac Lab's PhysX solver (NVIDIA's general-purpose rigid-body physics engine, originally built for games and now reused for robotics) make different approximation tradeoffs on deformable-object contact; teams working on cloth manipulation or soft-body grasping need an extensible solver interface that neither engine exposes. Newton targets all three gaps at once. Figure 11.5B lays out how these three pieces fit together architecturally: OpenUSD scenes feed Newton's solver core, which Warp compiles into GPU kernels that ultimately hand environment observations to policy training.

The useful mental model is that Newton is infrastructure for the class of tasks where parallelism, USD-native scene graphs, and solver extensibility are simultaneously load-bearing. For a Franka Panda pick-and-place study with a fixed tabletop and fewer than 4,096 parallel rollouts, MuJoCo or Isaac Lab already covers the requirement and carries years of benchmark validation. Newton's validation bar is correspondingly higher: before treating it as a dependency, verify solver behavior on your specific contact geometry, measure step throughput against a MuJoCo Warp baseline on the same GPU, and confirm that your USD assets arrive with physics schemas intact rather than engine-default friction values.

Here is a number worth pausing on: in the 2025 Newton launch benchmark, scaling from 4,096 to 32,768 parallel environments on a single A100 produced only a 12% drop in per-environment step throughput, because Warp kernels fill GPU streaming multiprocessors (the independent compute units on a GPU die that execute thousands of threads in parallel) rather than OS threads. A CPU-threaded simulator running the same expansion would have saturated available cores and degraded nearly linearly. That 12% is the concrete argument for the architecture.

Warp kernels matter because policy training requires millions of environment steps, and each step solves contact constraints across every joint and collision surface. A CPU-threaded simulator serializes those solves across cores. A Warp kernel writes the constraint equations as a GPU-compiled function that runs simultaneously across tens of thousands of environments, each on its own GPU thread. The consequence for real robots is direct. More rollouts in the same wall-clock time let a policy cover more of the physical state space before deployment. That coverage shrinks the gap between simulator behavior and on-hardware behavior, the gap that makes controllers fail on unexpected terrain or grasp angles. To put that in concrete terms: a locomotion policy trained on 4,096 CPU-threaded environments for 12 hours sees roughly the same number of unique contact configurations as one trained on 32,768 GPU-parallel environments for under 90 minutes, so the architectural choice does not change how much data the policy needs, only how long you wait for it.

Mechanically, Warp compiles Python-annotated functions into CUDA kernels at import time using a JIT (just-in-time) step, meaning the Python code is translated into compiled GPU instructions the moment it is first loaded rather than ahead of time. Each kernel receives per-environment state buffers, performs the physics integration (integrating velocities, resolving contacts via the chosen solver, updating positions), and writes results back to GPU memory without transferring data to the CPU. Newton schedules these kernels across the environment batch, so the physics step time scales with the number of GPU streaming multiprocessors rather than with the number of CPU cores.

OpenUSD As More Than A File Format

Warp solves the throughput half of Newton's design; the other half is the question of where the scenes those kernels simulate actually come from, and that is where OpenUSD enters.

OpenUSD matters because embodied AI scenes keep growing more compositional. A single household robot scene bundles rooms, articulated objects, materials, lights, cameras, semantic labels, robot assets, and task-specific randomization, and a scene graph makes every one of those components reusable. Physics is only half the requirement: the research workflow also needs interchange between tools.

The Physics Schema Gap

Scene interchange is not the same as physical validity. A USD asset can preserve hierarchy, materials, and semantics while still leaving collision geometry, inertial properties, joint limits, or contact parameters under-specified for learning, a distinction that is called the physics schema gap, and it is responsible for most silent failures when graphics assets enter a simulator. Treat OpenUSD as the scene backbone, then audit which simulator-specific assumptions get attached before the policy sees the world.

To make this concrete: the OpenUSD Physics schema (part of the USD specification since version 21.11) defines prim types such as PhysicsRigidBodyAPI, PhysicsCollisionAPI, and PhysicsMassAPI that attach mass, friction, and restitution values directly to scene prims. A Newton or Isaac scene that fully populates these schemas can be exchanged between tools without re-specifying contact parameters. A scene imported from a graphics pipeline that skips these schemas will arrive in the simulator with engine-default values, which for contact-rich tasks can mean friction coefficients of 0.5 instead of the intended 0.8, silently changing grasping or locomotion behavior.

The audit procedure below makes that inspection systematic: it walks every prim, compares any declared physics values against the task's intended contact parameters, fills documented gaps, and runs a single-step dry simulation as a final sanity check before issuing a go/no-go decision.

A Recency-Aware Adoption Score

Those decision questions are easy to nod along to and easy to forget under deadline pressure, so the next step is to turn them into a number that forces the tradeoff into the open.

Code Fragment 1 implements a tiny adoption checklist. It does not decide for you. It forces a team to separate frontier value from project risk before adding a new engine to a research dependency chain.

# Newton simulator adoption scorecard: weighted criteria scoring
# Demonstrates recency-aware decision logic for frontier physics engine selection.

import numpy as np

criteria = {
    "usd_scene_interchange": {
        "weight": 0.20,
        "description": "Project uses OpenUSD scene graphs for asset composition",
    },
    "custom_gpu_kernels": {
        "weight": 0.20,
        "description": "Task needs Warp-based custom physics kernels",
    },
    "open_governance": {
        "weight": 0.10,
        "description": "Linux Foundation stewardship reduces vendor lock-in risk",
    },
    "gpu_throughput": {
        "weight": 0.25,
        "description": "Need >1M physics steps/second on a single GPU node",
    },
    "mature_baseline_available": {
        "weight": -0.20,
        "description": "A mature engine (MuJoCo, Isaac Lab) already covers the task",
        "note": "negative weight: availability of a proven baseline raises the bar",
    },
    "reproducible_benchmark": {
        "weight": 0.05,
        "description": "Existing Newton benchmark matches project robot and task",
    },
}

# Score each criterion 0.0 (no) to 1.0 (yes, fully) for a hypothetical project.
project_scores = {
    "usd_scene_interchange": 0.8,
    "custom_gpu_kernels": 0.6,
    "open_governance": 1.0,
    "gpu_throughput": 0.9,
    "mature_baseline_available": 0.7,
    "reproducible_benchmark": 0.3,
}

total = sum(
    criteria[k]["weight"] * project_scores[k] for k in criteria
)

print("Newton Adoption Scorecard")
print("-" * 44)
for k, meta in criteria.items():
    contrib = meta["weight"] * project_scores[k]
    sign = "+" if contrib >= 0 else ""
    print(f"  {k:35s}  {sign}{contrib:+.3f}")
print("-" * 44)
print(f"  {'TOTAL ADOPTION SCORE':35s}   {total:.3f}")
threshold = 0.40
verdict = "Prototype Newton" if total >= threshold else "Stay with mature simulator"
print(f"  Threshold = {threshold}  =>  {verdict}")

Newton Adoption Scorecard
--------------------------------------------
  usd_scene_interchange                   +0.160
  custom_gpu_kernels                      +0.120
  open_governance                         +0.100
  gpu_throughput                          +0.225
  mature_baseline_available               -0.140
  reproducible_benchmark                  +0.015
--------------------------------------------
  TOTAL ADOPTION SCORE                    +0.480
  Threshold = 0.4  =>  Prototype Newton

When To Use Newton

Use Newton when you are exploring the frontier of GPU physics, OpenUSD-based robot scenes, custom solver work, or Newton-specific integration with Isaac Lab. Avoid making it the only simulator in a high-stakes study until you have a baseline comparison against a mature engine and a reproducible task artifact.

Consider a specific case (a hypothetical worked example, not a published benchmark): a team evaluating Newton for a bipedal walking task runs 4,096 parallel environments on a single A100 GPU and might plausibly measure 2.1 million physics steps per second at a 0.005-second timestep. They then reproduce the identical scene in MuJoCo Warp on the same hardware and record 2.8 million steps per second. The Newton run shows periodic contact-force spikes at uneven terrain seams that the MuJoCo Warp run does not exhibit. Without that side-by-side measurement, the team cannot tell whether Newton's contact solver is behaving correctly on their terrain mesh or whether a USD asset import left the collision geometry coarser than intended. The baseline comparison is what turns a throughput number into a trustworthy claim about the simulator.

Expected output: A Newton adoption note should include the exact repository or release, Warp and OpenUSD versions, the contact-rich task, the mature baseline, the metric, the solver settings, and the reason Newton changes the experiment rather than only the novelty of the stack.

Putting the decision questions, the adoption scorecard, and the schema audit together completes what this section promised at the outset: knowing what Newton offers, where its OpenUSD physics schema contracts break down, and how to decide whether it belongs in your infrastructure or your watch list. The decision procedure is: (1) run the audit algorithm above on your USD assets to get a go/no-go on schema completeness, (2) score the adoption scorecard for your project's criteria, and (3) if both clear their thresholds, run the baseline comparison against MuJoCo or Isaac Lab described earlier before committing Newton to a training pipeline. Skipping any one of the three steps is exactly the failure mode this section warns against: adopting a frontier engine on enthusiasm rather than evidence.