Physical AI, Chapter 5: The Data Bottleneck and Training Paradigms

This is Part 5 of a series walking through my book Physical AI: Engineering Embodied Intelligence for the Real World. In the previous chapter, we opened up the policy. This one asks where its training data comes from, and what it costs.

The central fact about data for Physical AI is that it is expensive, and the size of the gap is what makes it structural rather than merely annoying. A token of web text costs, for practical purposes, nothing; Common Crawl holds on the order of 10¹³ of them and was assembled by a small nonprofit with ordinary networking infrastructure. A second of teleoperated robot demonstration requires a paid human at a workstation, on hardware with a service life measured in hundreds of hours, followed by a second paid human to label and quality-control what was collected. An hour of good demonstration data costs a lab somewhere between one hundred and a thousand dollars.

Run the comparison out and it stops being a difference of degree. To assemble a robot dataset informationally comparable to a language pretraining corpus would mean roughly thirty thousand person-years of teleoperation and a bill in the billions. The largest thing the field has actually built — Open X-Embodiment, pooled across twenty-two embodiments and more than thirty labs — is about a million episodes. The ratio between what the field would like to train on and what it has is around four orders of magnitude.

Making the human hour go further

If data has to be collected by a person, the design of the rig is itself a research problem, and the chapter walks the ones that moved the field: Aloha, the twenty-thousand-dollar bimanual leader-follower design that put contact-rich data collection within reach of academic labs; Mobile Aloha, which put that upper body on a wheeled base and produced the household-task videos everyone saw in 2024; GELLO, which replaces the actuated leader arm with a cheap sensored puppet and gives up haptic feedback to do it; and UMI, the handheld gripper that records demonstrations with no robot present at all, at the price of reprojecting human wrist kinematics onto a robot arm.

The counter-intuitive finding from comparing them is that the hardware is not the dominant cost. The teleoperator’s time is. A rig that costs twice as much and doubles demonstrations per hour pays for itself in days. And quality beats quantity often enough to matter: a hundred clean demonstrations regularly outperform a thousand mixed ones, which is why DROID and BridgeData V2 both ship explicit quality metadata and why modern loaders have automated filtering built in.

Three paradigms, and the loop that compounds

Reinforcement learning owns locomotion, where the reward is easy to state and the simulator’s dynamics are close enough to transfer. Imitation learning owns manipulation, where the reward is not statable and exploration costs too much — with DAgger’s distribution-shift problem as the permanent hazard and action chunking, at 8 to 32 actions, as the compromise the field converged on. Foundation-model pretraining sits on top of both: pretrain on pooled data, fine-tune on a few hundred demonstrations of your platform.

Then there is the data flywheel, which is how deployed policies get better. Robots log their failures, teleoperators supply corrective demonstrations aimed exactly at those failures, the policy is retrained, and the cycle repeats. Its improvement rate is proportional to the failure-generation rate, which is proportional to fleet size — which is why the largest programs have been willing to scale fleets past the point where per-unit economics justified it. The fleet’s value is not only the work it does but the data it generates, and the advantage compounds at every turn of the wheel.

Tomorrow: most modern policy training does not happen on real robots at all. Chapter 6 is about simulation, the reality gap, and the techniques — domain randomization, domain adaptation, real-to-sim-to-real — built to cross it.