At CES 2025, NVIDIA announced the launch of Cosmos, an innovative platform designed to advance the development of physical AI—AI systems that interact with and understand the physical world. This cutting-edge platform will significantly accelerate the growth of autonomous vehicles (AVs), robotics, and other AI-driven physical systems by offering powerful world foundation models (WFMs), a state-of-the-art data processing pipeline, and advanced tools for developers.
Cosmos enables the creation of synthetic data and virtual environments for training AI models that will power robots and AVs, allowing for more scalable, cost-efficient, and safe AI model development. It provides developers with access to highly realistic, physics-based synthetic data, which can be used to train robots and AV systems in simulated environments before deploying them in the real world.
Democratizing Physical AI
NVIDIA’s new Cosmos platform introduces open access to world foundation models, which are designed to generate and predict realistic video simulations of real-world physical environments. These models utilize a variety of inputs—such as text, video, and sensor data—to create 3D physics-based environments, allowing developers to train AI agents without the significant costs of real-world data collection.
Unlike traditional AI models, Cosmos WFMs are capable of generating photorealistic synthetic data that is both dynamic and interactive, simulating complex real-world scenarios like warehouse operations, driving environments, and robotic tasks. These models are built for physics-aware video generation, object permanence, and spatial understanding—key for developing safe and robust physical AI systems.
In a statement, NVIDIA CEO Jensen Huang emphasized the transformative potential of Cosmos for the robotics and AV industries. “The ChatGPT moment for robotics is coming,” Huang said. “Like large language models, world foundation models are fundamental to advancing robot and AV development. Yet, not all developers have the expertise or resources to train their own. We created Cosmos to democratize physical AI and put general robotics in reach of every developer.”
Open and Scalable Platform for Developers
Cosmos WFMs will be available under an open model license, giving developers the flexibility to use them for commercial purposes. By accessing these models through NVIDIA’s NGC catalog and Hugging Face, developers can fine-tune the foundation models to meet the specific needs of their applications. The first wave of Cosmos models is already available, including diffusion and autoregressive models optimized for a range of tasks such as data curation, model training, and synthetic data generation.
Developers can generate high-quality synthetic data for use in testing autonomous driving scenarios, simulating industrial environments, or training robots to perform specific tasks. For instance, Cosmos can simulate complex driving conditions, such as icy roads, or create realistic warehouse environments for robotic training.
Key Features of Cosmos
- Physics-based Synthetic Data Generation: Cosmos can generate photorealistic video data for training, testing, and reinforcement learning in virtual environments, greatly reducing the need for expensive real-world data collection.
- Advanced Tokenizers: Cosmos includes a state-of-the-art video tokenizer that compresses and processes video data 12 times faster than current models, enabling more efficient training of AI models.
- NeMo Curator: A powerful tool for accelerated data curation, which processes and labels massive video datasets in a fraction of the time. With NVIDIA Blackwell GPUs, developers can process 20 million hours of video data in just 14 days, a task that would otherwise take over three years with traditional CPU-based systems.
- Multiverse Simulation: Using NVIDIA Omniverse, Cosmos allows developers to simulate every possible future scenario an AI agent could encounter, helping it to select the safest or most optimal path in real-world tasks.
Industry Adoption and Applications
Several major companies in the robotics and AV sectors are already adopting the Cosmos platform, including 1X, Agility Robotics, Waabi, and Uber, among others. These leaders in autonomous technology are using Cosmos to enhance their AI models with high-quality synthetic data and improve their physical AI systems.
- 1X, an AI and humanoid robotics company, is using Cosmos to develop its humanoid robots by augmenting training datasets with synthetic video data.
- Waabi, an AV company pioneering generative AI for autonomous vehicles, is evaluating Cosmos to accelerate the development of AV software through the curation of large-scale video datasets.
- Uber, the global ridesharing giant, is partnering with NVIDIA to build more advanced autonomous vehicles. By integrating Uber’s rich driving datasets with Cosmos, Uber aims to accelerate the deployment of safe and scalable autonomous mobility solutions.
These companies, along with others like XPENG and Foretellix, are leveraging Cosmos to create customized robotics and AV solutions by integrating synthetic data into their training and testing pipelines.
Trustworthy AI and Responsible Use
As part of its commitment to responsible AI, NVIDIA has ensured that Cosmos includes robust guardrails to mitigate harmful content and ensure the ethical use of AI. Videos generated by Cosmos include invisible watermarks, making it easier to identify AI-generated content, thus reducing the risk of misinformation or misattribution. The platform also features tools to enhance the accuracy of text prompts and prevent unwanted bias in the models.
“We are committed to trustworthy AI that prioritizes privacy, safety, and transparency,” said Huang. “Cosmos was built in line with these principles to ensure that developers have the tools they need to create safe and responsible AI systems for the future.”
A New Era for Physical AI Development
With the launch of Cosmos, NVIDIA is positioning itself at the forefront of the next generation of physical AI. By providing developers with the tools and resources to generate synthetic data, create custom models, and accelerate the training of robots and AVs, Cosmos is set to revolutionize industries from robotics to autonomous vehicles.
The first wave of Cosmos models is available now through the NGC catalog and Hugging Face, with more models and features slated for release soon. Developers can also take advantage of NVIDIA DGX Cloud for seamless deployment and enterprise support.
As the AI industry continues to evolve, NVIDIA’s Cosmos platform offers the foundation for the next wave of AI-powered robots and autonomous vehicles—unlocking new possibilities for developers and accelerating the deployment of intelligent systems worldwide.
