OpenAI drops free MuJoCo alternative with 12 robot sims

The paid license requirement for MuJoCo has been a quiet thorn in the side of robotics researchers for years. OpenAI just pulled it out. They’re releasing Roboschool, an open-source simulation package built on the permissively licensed Bullet Physics Engine, and it’s shipping with twelve ready-to-go environments integrated directly into OpenAI Gym. Eight of those are free, re-tuned alternatives to existing MuJoCo tasks. The team didn’t just port them—they claim the new versions produce more realistic motion than the originals.

But the real signal here isn’t just the price tag dropping to zero. It’s the new environments designed to break the brittle, single-purpose walking policies that plague standard benchmarks. Anyone who has trained a standard humanoid walker knows the dirty secret: you get a fragile, cyclic gait that shatters the moment you look at it wrong. Roboschool’s HumanoidFlagrun and HumanoidFlagrunHarder force the agent to run toward a randomly moving target. The harder version adds a barrage of white cubes to knock the robot off course and starts each episode with the bot flat on the ground. The trained policies look clunky and unnatural, but they can steer and recover from falls. OpenAI notes something fascinating here: the policy is a stateless multilayer perceptron, so they suspect the agent is using its arm positions to store information between timesteps.

The package also introduces native multi-agent training, starting with RoboschoolPong. This is where things get delightfully messy. If you train two agents simultaneously with policy gradients, they fall into a useless cat-and-mouse loop. One agent learns to camp at the top, so the other overfits to shooting low. Eventually the first moves down, and they oscillate forever without learning anything robust. It’s the same maddening dynamic you see in generative adversarial networks. OpenAI isn’t framing this as a failure—they see it as a feature. The adversarial interplay could naturally generate a curriculum for increasingly sophisticated strategies, and they’re explicitly calling it out as a research problem worth tackling.

The code is on GitHub with pre-trained policies in an agent_zoo folder and a demo script to race three robots against each other. For labs and students who couldn’t justify MuJoCo’s license—even with the recent free student tier—this removes a real barrier. The Bullet engine underpinning it all already powers Gazebo and V-REP, so this isn’t some fragile toy simulator. It’s a legitimate research tool that just happens to be completely free.