NVIDIA launches customizable Cosmos models for physical AI

NVIDIA has announced the release of new Cosmos™ world foundation models (WFMs), designed for physical AI development. This update allows developers to customize reasoning models for better control over world generation. The company introduced new blueprints supported by the NVIDIA Omniverse™ and Cosmos platforms. These blueprints provide developers with tools to create large-scale synthetic data for robots and autonomous cars more efficiently. Leading companies like Uber and Agility Robotics are among the first to use Cosmos for enhancing training data. NVIDIA's CEO Jensen Huang highlighted that just as large language models changed generative AI, Cosmos models represent a significant advancement for physical AI. This development aims to improve robotics and related industries. Cosmos Transfer WFMs can create realistic video outputs from structured video inputs like depth maps and lidar scans. This process helps streamline AI training by converting simulations into high-quality videos. Agility Robotics plans to use Cosmos for training their robot models. The NVIDIA Omniverse Blueprint for autonomous vehicle simulation enhances sensor data by varying conditions, like weather, to create diverse datasets. Another blueprint, the NVIDIA GR00T, reduces the time needed for data collection, turning days of work into hours. Cosmos Predict WFMs can generate virtual world states from various inputs including images and videos. These models enable prediction of motion trajectories, making them useful for improving AI systems like robots and autonomous vehicles. Developers can also customize Cosmos Reason, which uses reasoning to predict outcomes for interactions in video data. This model can enhance data annotation and help create plans for AI tasks. NVIDIA provides developers with tools for data curation and post-training, making it easier to process large datasets for training AI models. Notable companies are already utilizing these tools for various applications. To support responsible AI, NVIDIA is implementing guardrails for all Cosmos WFMs and partnering with Google DeepMind to help watermark AI-generated outputs. The new Cosmos WFMs are available for preview in the NVIDIA API catalog and can also be found on Google Cloud's Vertex AI Model Garden.