NVIDIA's new surgical simulator runs in real time on a single GPU

NVIDIA just turned a research demo into something surgeons can actually interact with. Cosmos-H-Dreams is a real-time, AI-driven simulator for surgical robotics that runs on a single RTX PRO 6000 GPU. It’s a distilled, streamlined student model that descends from the company’s earlier Cosmos-H-Surgical-Simulator, but unlike its offline predecessor, this one reacts to a surgeon’s or an AI policy’s actions as they happen.

The core trick isn’t just shrinking the model—it’s a training process at NVIDIA called self-forcing distillation. Autoregressive models that generate video frame by frame tend to drift into uncanny territory because they train on perfect data but have to eat their own imperfect outputs during deployment. NVIDIA’s pipeline forces the student model to learn from its own generated history, with a frozen teacher model constantly course-correcting the output so it stays grounded in realistic surgical dynamics instead of veering off into AI-generated nonsense. The team also fine-tuned the teacher on failure cases like dropped needles and missed sutures, because a simulator that can’t model bad outcomes is useless for training.

The result is a model that uses as few as two denoising steps per frame and streams video autoregressively through NVIDIA’s new FlashDreams inference library. The demo is trained on dVRK tabletop suturing data from JHU and maps the robot’s arm movements directly to visual predictions. NVIDIA also collaborated with CMR Surgical and Cambridge Consultants to get it working with the Versius surgeon controller, suggesting this isn’t just a sandbox project—it’s built to plug into real hardware setups. A simulator that can run faster than physical time on a workstation GPU changes the economics of surgical AI training, making it possible to test and iterate on policies without tying up a multi-million dollar robot in a wet lab.