World models and sim2real

What is a world model?

A world model is a model that learns the dynamics of the world: given observations and actions, it predicts what happens next. It gives the machine a form of imagination: testing an action in thought before executing it, or generating situations it has never encountered.

Two families coexist. Generative video models predict future pixels directly: NVIDIA Cosmos, Google DeepMind's Genie 3, which generates interactive worlds navigable in real time at 24 frames per second in 720p, or Wayve's GAIA series for driving. Latent predictive models, such as the JEPA family championed by Yann LeCun, predict in an abstract representation space rather than in pixels: less spectacular, but cheaper and potentially better suited to reasoning.

The concrete uses in robotics: generating synthetic training data, evaluating a policy inside a learned environment before deploying it, and serving as a planning module inside the robots themselves.

Yann LeCun's approach: JEPA and AMI Labs

Yann LeCun, a deep learning pioneer and 2018 Turing Award laureate, has spent several years arguing an iconoclastic thesis: autoregressive large language models, however powerful, do not lead to human-level intelligence. An LLM trained to predict the next token learns a model of language, not a model of the world: no persistent memory, no real planning, reasoning he himself calls very primitive. On X and then on LinkedIn, he summarized his position with an image that has since been widely quoted: "on the highway towards Human-Level AI, Large Language Model is an off-ramp." He has gone as far as advising PhD students not to work on LLMs. His underlying argument: a system trained purely on text learns a model of words, not a model of the physical world, and therefore cannot acquire the common sense or the capacity to act that a child develops within months of interacting with its environment.

His architectural answer is called JEPA, Joint Embedding Predictive Architecture. Introduced in his 2022 position paper "A Path Towards Autonomous Machine Intelligence," the idea is to predict not future pixels or tokens, but an abstract representation of that future, in an embedding space. JEPA is an energy-based model: two encoders project the observed context and the target to predict, and a predictor links the two representations without reconstructing the raw observation, which lets the model ignore unpredictable detail and keep only the semantic structure of the signal. The lineage rolled out in stages: I-JEPA for images (CVPR 2023), V-JEPA for video (February 2024), then V-JEPA 2 (June 2025), pretrained on more than a million hours of video and fine-tuned on only 62 hours of robot data. V-JEPA 2 plans pick-and-place actions on objects and in environments it has never seen, zero-shot, reaching success rates of 65% to 80% depending on the task, using its world model for predictive control rather than a fixed policy.

In November 2025, after more than a decade leading Meta's FAIR lab, Yann LeCun left the company to found Advanced Machine Intelligence Labs (AMI Labs), alongside Alexandre LeBrun, a former FAIR Paris researcher who went on to co-found the health-tech startup Nabla. LeCun serves as executive chairman, LeBrun as CEO. In March 2026, AMI Labs raised $1.03 billion in a seed round at a $3.5 billion valuation, the largest seed round on record in Europe, led by Cathay Innovation, Greycroft, Hiro Capital, HV Capital and Bezos Expeditions, with participation from NVIDIA, Temasek, Samsung, Toyota Ventures and Bpifrance. Headquartered in Paris, with offices in New York, Montreal and Singapore, the company is aiming to build JEPA-based world models for industry, robotics and healthcare.

These three families sit on different maturity horizons. VLAs are already deployed: they are what drive arms and humanoids today. Generative world models, Cosmos and GAIA foremost, are also operational, mainly as a data factory and simulation tool upstream of deployment. JEPA remains the longest-term bet: compelling in research, promising in early robotic control demonstrations, but still far from an industrial product on a par with VLAs already in service. For an industrial buyer deciding where to invest in 2026: bet on VLAs for near-term deployments, follow generative world models for synthetic data, and keep an eye on JEPA and AMI Labs as a mid-to-long-term option. VLAs already in production are listed in our models comparator.

The role of simulation

Before learned world models, there is classical physics simulation, which remains the workhorse of robotics. The reference simulators: Isaac Sim and Isaac Lab at NVIDIA, MuJoCo at Google DeepMind, and Genesis on the open-source side. Their decisive advantage is GPU parallelism: thousands of simulated environments running in parallel accumulate years of experience in a few hours, which is what made reinforcement learning of locomotion policies possible. Our simulators comparator details licences, physics engines and ROS support for these tools.

The field's big 2026 event is Newton, an open-source physics engine built on NVIDIA Warp and OpenUSD by NVIDIA, Google DeepMind and Disney Research, and contributed to the Linux Foundation in late 2025. Version 1.0 shipped at GTC 2026. Newton notably includes MuJoCo Warp, which speeds up manipulation tasks by more than 70x over classic MuJoCo, and differentiable physics that lets gradients propagate through the simulation.

The sim2real gap and how to cross it

A policy that is perfect in simulation can fail on the real robot: this is the sim2real gap. Friction, contacts, deformable objects, cables and lighting are all simulated imperfectly, and the model learns to exploit the simulator's approximations rather than real physics.

Three main techniques help cross it.

• Domain randomization: randomly varying masses, friction, textures and lighting during training, forcing the policy to become robust to a whole range of possible physics, of which reality is one instance.
• System identification: calibrating the simulator against measurements from the real robot to shrink the gap at the source.
• Real-world fine-tuning: finishing training with a short phase on the physical robot, using teleoperated data or reinforcement learning.

World models add a new route: Cosmos Transfer re-renders simulated scenes as photorealistic images to shrink the visual gap. And the approach keeps improving: Mistral AI's Robostral Navigate, trained entirely in simulation, is designed to deploy on real wheeled, legged or flying robots without prior field data collection.

The data engine

Serious labs do not collect data once: they build a data engine, a continuous loop in which deployed robots report their successes and failures, which feed the next round of model training. Three streams feed this loop.

Teleoperation remains the quality benchmark: a human operator drives the robot, and every trajectory is a demonstration perfectly aligned with the target embodiment. But it is expensive and saturates quickly. Egocentric human capture changes the equation: filming humans performing tasks with head-worn cameras produces far greater volumes. Work published in 2026 shows that 30 minutes of human video per task can outperform an equal amount of teleoperation time by 41%. Finally, synthetic generation by world models (NVIDIA's Cosmos Predict) multiplies the variations of any given scene.

The dominant recipe in 2026: pretrain on human video and synthetic data, fine-tune with targeted teleoperation, then learn continuously from deployments.

Key players in 2026

Four players structure the world model landscape in 2026.

• NVIDIA launched Cosmos 3 in June 2026, billed as an open frontier foundation model for physical AI. Its mixture-of-transformers architecture couples a reasoning module with a generation module, in service of synthetic data production for robots and autonomous vehicles.
• Google DeepMind opened Genie 3 to the public through Project Genie in January 2026 (AI Ultra subscribers in the US); Waymo derived from it a dedicated world model for simulating edge cases for its robotaxis.
• Meta and AMI Labs: Meta carried the V-JEPA research line before Yann LeCun's departure to found AMI Labs. The company raised $1.03 billion in March 2026, Europe's largest seed round to date, to build JEPA-based world models, with headquarters in Paris.
• Wayve followed GAIA-2 with GAIA-3 in December 2025: a 15-billion-parameter world model dedicated to large-scale offline evaluation of autonomous driving systems.

Keep reading

• See how the VLA consumes this synthetic data in our VLA models explained guide.
• Place simulation within the wider stack with our physical AI stack pillar.
• Compare the simulators that produce this data in our simulators comparator.