Induction Labs’ 106B model masters tasks from raw video, no action labels needed
Curated by the Inblix editorial team
The standard recipe for training AI agents from video has a stubborn requirement: you need to know the action that caused each frame. Induction Labs just threw that assumption out. They’ve released Photon-1, a 106-billion-parameter mixture-of-experts transformer that pretrains on raw, unlabeled video and still learns to complete tasks. The secret is what they call an imagination model. It predicts future frames in a compressed latent space, never touching pixels during pretraining. The model develops what the team describes as an implicit policy—a grasp of what a person is doing without ever seeing a single mouse-click label.
The numbers behind this are genuinely unusual. Photon-1 was trained on 18 years of computer demonstration video, or 575 million frames, compressed to just 2.2 KB each using a finite scalar quantization encoder. That’s over 100× better compression than existing multimodal representations, and it preserves text, layout, and state changes. The entire pretraining run consumed roughly 30,000 H200 GPU-hours at 40% model FLOPs utilization. On an internal computer-use benchmark, Photon-1 beats Google’s Gemini 3.1 Flash-Lite while costing about 3× less to serve. Induction Labs estimates it used at least 30× less pretraining compute, though that number leans on their own conservative math for what Gemini required.
Finetuning is where the imagination collides with the real world. With fewer than 35,000 demonstration trajectories, the team taught Photon-1 to emit actions using special computer-use tokens. At inference, the model predicts the next frame’s state first, then outputs the action to reach it. Online reinforcement learning follows on fleets of Linux VMs running five different desktop environments, with outcomes verified programmatically. It’s a tight feedback loop that doesn’t depend on human raters.
Does any of this transfer beyond desktop video? Early signs say yes. On 20,000 tournament checkers games and 10,000 synthetic billiard simulations—domains completely absent from pretraining—Photon-1 beat both a same-sized vision encoder baseline and a 20-trillion-token LLM baseline on world simulation and move quality. The billiard physics error dropped to 0.47 mean absolute error against the ground truth, versus 1.44 for the vision encoder. The model also picked up some strangely human behaviors, like actively using an in-VM chatbot to draft answers during knowledge tasks. Whether that’s clever or alarming depends on your tolerance for agents that learn to delegate.
💡 Key Takeaways
- Photon-1 proves that predicting future video frames without action labels creates a useful implicit policy for completing real computer tasks.
- Induction Labs’ differential latent encoder compresses video frames to 2.2 KB each, preserving critical details at over 100× better rates than existing multimodal representations.
- The model required only 30,000 H200 GPU-hours for pretraining, undercutting larger commercial models on both compute and per-token serving cost by a wide margin.
- After finetuning on just 35,000 trajectories and running online RL at scale, Photon-1 learned to use an internal chatbot as a tool—a behavior that emerged from pretraining priors, not explicit instruction.
Keep reading: See related articles below for more coverage on this topic.
Get smarter about AI
The sharpest AI news, curated daily. Delivered free to your inbox.