MirroS just dropped a genuinely different idea: stop treating video as pixels to predict, and start treating it as evidence of an underlying physical program. Their Code-as-World framework represents scenes as executable world representations — a scene.json file that MuJoCo can actually run. The triple structure is clean: Composition (objects, mass, friction, gravity), Evolution (forces, contacts, termination conditions), and Appearance (camera, lighting, materials). Change the appearance and the physics stays untouched. That separation is the kind of thing that makes engineers nod.
The recovery process is agentic, not one-shot. An agent proposes a world program, instantiates it, renders it, and verifies against the source video across RGB, depth, masks, and trajectories. SAM 3 supplies masks, VGGT-Omega handles camera geometry, and SAM 3D generates meshes. Up to five rounds of propose-verify search, with structured feedback guiding each revision. Here’s the sharp result: at a matched five-evaluation budget, this loop beats Best-of-5 independent sampling on every metric they tested — Visual Alignment, Object IoU, Traj-ADE, Accuracy@2%D. That’s not a marginal win; it’s evidence the feedback loop is doing real work.
The training pipeline turns verified worlds into supervision. Phase 1 fine-tunes on 73,335 image-space QA pairs from RefCOCO and GOT-10K. Phase 2 applies GRPO to world-space VQA from 1,585 text-driven and 988 video-driven worlds, rewarded on scale-normalized numerical accuracy. The ablation tells the story: image-space-only scores 50.9 on the 9B; adding world-space RL lifts it to 55.4. On QuantiPhy-validation, that beats Gemini-3.1 Flash at 54.8 and crushes Qwen3-VL-32B-Instruct at 40.2. Pixel grounding improves too — RefCOCO jumps 63.7 to 68.3.
The honest caveats: rigid-body physics only, and the model never learns the discovery loop itself — that stays external. This is a research artifact, not a product. But the framing is the kind of thing that could age well: when you need physical understanding, pixels are a lossy compression. Code might be the better representation. Both checkpoints are Apache 2.0, fine-tuned from Qwen3.5, served via vLLM. Go break it.
