NVIDIA Cosmos3-H-Dreams: Bringing Clinical Surgery to Life Through Generative Simulation

NVIDIA has introduced Cosmos3-H-Dreams, an interactive generative simulation for creating realistic, real-time clinical surgery environments. This world foundation model learns visual dynamics directly from synchronized video and robot kinematics from actual cholecystectomy procedures, avoiding explicit physical modeling. The system is built on the Cosmos 3 foundation model and uses a teacher-student distillation pipeline to create an efficient model for sequential video generation. Accelerated by the FlashDreams streaming inference library, Cosmos3-H-Dreams can run on a single GPU and generate coherent surgical scenes that respond to live kinematic control, even for novel movements not present in the training data.