LLMs trained on code just learned to evolve robots from scratch

Here’s something you don’t see every day: a large language model that’s never laid eyes on a walking robot suddenly generating hundreds of thousands of functional ambulating machines. That’s the headline result from a new paper introducing Evolution through Large Models, or ELM, which shows that code-trained LLMs can supercharge genetic programming in ways that shouldn’t be possible on paper. The authors fed an LLM the task of mutating programs in Sodarace, a physics simulation domain the model had zero exposure to during pre-training. The payoff? A staggering diversity of working walkers — and not just a handful of lucky flukes, but hundreds of thousands of them.

The secret sauce is that LLMs trained on code repositories have internalized something surprisingly useful: the patterns of incremental changes that human programmers make. When you treat those models as mutation operators inside a genetic algorithm, they stop making random syntactic noise and start proposing edits that actually make sense. Combine that with MAP-Elites, a quality-diversity algorithm, and the system starts exploring not just what works, but what works in meaningfully different ways. One terrain might need long legs, another a low center of gravity — and ELM finds both without being told either.

What’s genuinely new here is the bootstrapping move. Those hundreds of thousands of robot-morphology programs aren’t just a cool demo. They become a training set for a fresh conditional language model that learns to map terrain types to appropriate robot designs. Think about that: from zero domain-specific data to a generative model that can produce the right artifact for a given context. That’s a closed loop that smells like something important — a mechanism for building domain expertise where none previously existed.

I’ll admit, the open-endedness framing always makes me squint a little — it’s a term that gets thrown around like confetti at a parade. But if you can take an LLM, drop it into a domain it’s never seen, and watch it produce not just solutions but a whole taxonomy of them that then trains another model, you’re creeping toward something that deserves the label. The implications spill across deep learning and reinforcement learning, sure, but the paper’s most provocative idea is simpler: what if the path to general-purpose mutation is just a language model that’s read enough commit histories?