How robots learn now

Something changed in the robot demos, and it's worth being precise about what. A robot that cleans a kitchen it has never seen, from a single spoken instruction. A robot that operates an appliance that appeared twice in its entire training set. These aren't scripted motions replayed with good production values — they're learned behavior, produced by the same recipe that built the language models: big models, big data, general pretraining, narrow post-training.

This note is a map of that recipe as it stands in mid-2026 — the stack, the pipeline that trains it end to end, and the places it's stuck. The field moves fast enough that the details have a shelf life; the shape should hold longer.

The stack

The modern robot is three layers, and the order of difficulty has inverted.

The body. Actuators, sensors, hands. For decades this was the bottleneck; it mostly isn't anymore. Capable arms and humanoid platforms now come off supply chains at steadily falling cost, and for the majority of tasks the hardware can already do physically what the software cannot yet tell it to do. The stubborn exception is dexterity — hands that match human touch and compliance remain genuinely hard — but the industry's center of gravity has moved decisively from building bodies to building brains.

The brain. The unit of progress here is the vision-language-action model, the VLA. The trick that defines the current era: don't train a robot brain from scratch — start from a vision-language model that has already absorbed the internet, so the robot inherits object knowledge, language, and a rough common sense of the world for free, then attach an action head that turns all of that into motor commands. Two schools of thought compete on that last step: treat actions as tokens and generate them the way an LLM generates text, or generate continuous trajectories with diffusion or flow matching. The frontier systems — Physical Intelligence's π series, Google DeepMind's Gemini Robotics 2, NVIDIA's open GR00T models — are converging on a split architecture: a slow reasoner that reads the scene and plans in something like language, and a fast controller that executes at real-time rates. Cortex and cerebellum.

The world. The newest layer, and 2026's breakout: models of the environment itself. Physics simulators have trained locomotion policies for years; what's new is generative world models — NVIDIA's Cosmos family, DeepMind's Genie line — that produce photorealistic, interactive environments on demand. A policy can now practice in a thousand generated kitchens before it touches one real one, and the visual realism is starting to genuinely close the gap that made "trained in sim" a warning label.

The pipeline

End to end, a frontier robot policy in 2026 is built in roughly five moves.

Inherit the internet. Training starts from a pretrained vision-language model. This is where the robot gets everything that isn't motion: what a mug is, what "put it away" means, what usually happens next.

Mid-train on everything that moves. Then the model learns motion from three sources at once, and the field has stopped arguing about which one wins, because the answer is a portfolio. Teleoperation — a human piloting the robot — is the gold standard: real robot, real actions, real contact physics. It is also brutally expensive per hour. Simulation is nearly free at the margin and infinitely scalable, but pays a realism tax. And human egocentric video — head-mounted cameras recording people simply doing tasks with their own hands — is the wildcard that matured this year: no robot in the loop at all, which means no action labels, but staggering diversity at commodity cost. The year's most important result is that scaling laws finally showed up here: train on tens of thousands of hours of action- labeled human video and policy quality improves log-linearly, and at the margin an hour of human video now appears to buy more generalization than an hour of teleoperation. The lesson of the LLM era — the data engine is the moat — has arrived in robotics intact.

Post-train for the job. The generalist base model then gets the LLM-style treatment: supervised fine-tuning on curated demonstrations for a specific robot and task family, and increasingly reinforcement learning on top — in simulation, and now cautiously on real hardware — to push past the ceiling of pure imitation. Imitation gets you what the demonstrator did; RL gets you what the demonstrator would have done on the bad days, which is where recovery behavior lives.

Rehearse in the world model. Before a policy touches hardware, it can now be rolled out inside generated environments — varied lighting, varied clutter, varied physics — both to harden it and to evaluate it cheaply. This is the layer moving fastest, because it attacks two bottlenecks at once: data scarcity and evaluation cost.

Compress and deploy. Finally the model is distilled and quantized onto onboard compute, with the reasoner running slow (and sometimes off-board) while the controller holds a hard real-time loop. Making frontier-scale cognition fit a power budget measured in hundreds of watts is its own active research front.

The roadblocks

Four problems absorb most of the industry's effort right now.

There is no internet of actions. Language models had the web waiting for them; robotics has to manufacture its equivalent. That's what the three-source data portfolio really is — a workaround for a corpus that doesn't exist. The scaling-law results are encouraging precisely because they suggest the workaround compounds, but the total volume of action data on earth is still a rounding error against what text models trained on.

Evaluation is the quiet crisis. Simulation benchmarks saturate and stop predicting reality; real-robot evaluation is slow, expensive, and unstandardized, and a "92% success rate" measured in one lab means little in yours. Distributed real-world evaluation efforts are nascent. Until the field can measure progress cheaply and comparably, it will keep mistaking demo reels for capability curves — you cannot improve what you cannot measure at scale.

The last nines. The gap between a 90% demo and a deployable system is not 10%; it's the entire product. Errors compound over long-horizon tasks, and industrial work is priced in nines — the Figure pilots on BMW's production line crossed 99% on a deliberately narrowed task, which is precisely the point: today, reliability is bought by narrowing scope. The frontier's open question is whether generality and nines can be had at the same time, and recovery — noticing a failure and fixing it — matters more to that answer than peak skill does.

Generalization is real but shallow. The new models transfer to novel objects and instructions in ways that would have seemed like science fiction three years ago — and still stumble on a lighting change, an odd camera angle, a layout no one anticipated. Certifying a learned, whole-body policy as safe around people is a discipline that barely exists yet. It will need to, before the demos become shifts.

The gap is the opportunity

None of these roadblocks reads like a wall; they read like a work list — and the pattern of the last three years is that this field clears its work list faster than anyone's roadmap predicted. What that means practically: the intelligence being built at the frontier will not arrive on factory floors all at once. It arrives in layers — perception first, then narrow skills, then autonomy — through the systems already running there. That's the seam we build in: the same substrate that dispatches a software agent today is built to dispatch a robot tomorrow, and the distance between those two sentences is exactly the distance this field is closing.