Nvidia is making strides in the world of artificial intelligence with the launch of their Cosmos World Foundation Models (Cosmos WFMs). These innovative AI models are inspired by how humans naturally develop mental models of the world. At the CES 2025 event in Las Vegas, Nvidia announced this exciting development: a family of world models that can predict and create “physics-aware” videos, designed to revolutionize simulation and data generation.
The Cosmos WFMs are part of Nvidia’s broader strategy to make these tools widely accessible. They’re available through Nvidia’s API and NGC catalogs, on GitHub, and the popular AI development platform, Hugging Face. This initiative opens up possibilities for researchers and developers, regardless of the company’s size, under a permissive open model license that allows for commercial exploitation.
These models are categorized into three tiers: Nano for low-latency, real-time applications, Super for high-performance baseline models, and Ultra for those demanding maximum quality and fidelity. The varying sizes, from 4 billion to 14 billion parameters, offer flexibility and adaptability across different applications. More parameters typically signify better performance in problem-solving tasks.
Nvidia has also introduced complementary tools as part of the Cosmos WFM package, including an upsampling model optimized for augmented reality and guardrail models to ensure responsible use. Fine-tuned models aimed at generating sensor data for autonomous vehicle development are also on offer. All Cosmos WFMs have been trained on an extensive dataset, encompassing 20 million hours of real-world interactions gathered from various domains such as robotics, industry, and driving.
Despite concerns about the origin of this training data, Nvidia assures that Cosmos WFMs are developed within legal guidelines, emphasizing that they learn factual world knowledge, which isn’t subject to copyright protections.
One of the standout features of the Cosmos WFM is its ability to simulate realistic environments like factory floors or navigate a warehouse, augmenting training for robots and autonomous vehicles. Industry leaders like Uber, Waabi, Wayve, and Fortellix have already started piloting these models for various applications, including building AI systems for autonomous driving and advancing video search and curation technologies.
Although not open-source in the traditional sense, due to incomplete transparency on training data and tools, Nvidia’s Cosmos WFMs are labeled as “open,” highlighting Nvidia’s commitment to pushing the frontiers of robotics and industrial AI.
Through these world models, Nvidia aims to accelerate the future of AI-driven mobility and industrial automation, echoing the impact enterprise models like Llama have had on the tech industry. This leap forward promises to transform how synthetic data is used to develop more sophisticated AI systems, paving the way for groundbreaking innovations in multiple sectors.
