NVIDIA Cosmos 3 Four Open Checkpoints for Driving and Robot World Simulation
Posted As of
NVIDIA has added four open-weight checkpoints for interactive world simulation to the Cosmos 3 family. They target multi-sensor driving scenes and bimanual robot simulation, and are not general-purpose text-to-video models.
What happened
The information is based on a post NVIDIA published on the official Hugging Face blog.
- Four open-weight checkpoints have been added to the Cosmos 3 family. They are intended for interactive world simulation and multi-sensor, multi-view driving scenes.
- Cosmos3-Nano-Transfer-Auto generates multi-view driving data with changed weather, lighting, and time of day.
- Cosmos3-Nano-Sim-Auto streams synchronized camera and LiDAR observations for driving simulation.
- The bimanual simulation checkpoint supports YAM and AgiBot embodiments and is controlled either by a policy or by a human using a joystick.
- All four are fine-tuned examples of Cosmos 3 Nano (16B trainable parameters, official), and are reported to be usable for commercial and non-commercial purposes within the terms of the OpenMDW1.1 license.
Our take
These are better viewed as materials for physical AI simulation than as models for direct local video generation. Closed-loop use reportedly requires a separate simulation runtime and a WSM renderer, so this is not a tool you can use just by installing it. Within the range I was able to confirm, the name of the fourth checkpoint, benchmark figures, clip length, and resolution were not given. It is also unknown what hardware specs are needed to run it on a Mac or a typical PC. The base Cosmos 3 was released in June, and these four are the only new additions this time.
Sources and related posts
- NVIDIA on Hugging Face Blog, New NVIDIA Cosmos 3 Open Models Primary source