Meta-learning gets curious: E-MAML and E-RL² tackle exploration

The hardest problems in reinforcement learning aren’t about mastering a known skill — they’re about figuring out what to do when you’re dropped into a situation you’ve never seen before. Bradly Stadie and a team from UC Berkeley and OpenAI are taking a swing at this with two new algorithms, E-MAML and E-RL², designed explicitly to make meta-learned agents better explorers. Their paper, posted on arXiv, doesn’t just rename existing ideas. It bakes the drive to explore directly into the meta-learning objective itself.

Here’s what that looks like in practice. Traditional MAML (Model-Agnostic Meta-Learning) trains a model to be just a few gradient steps away from solving a new task. The twist with E-MAML is that the “task” isn’t just about maximizing reward — it’s about maximizing information gain. The agent learns to actively seek out states that reduce its uncertainty about the world. E-RL² does something similar but operates on a different backbone, using a recurrent policy that adapts its behavior over time. Both methods were tested on a new environment the researchers call “Krazy World” and a suite of maze tasks, where the payoff for poking around is immediate and obvious.

The Krazy World results are where things get interesting. Standard meta-RL agents tend to freeze up or repeat known behaviors when faced with genuine ambiguity. E-MAML and E-RL², by contrast, exhibited systematic exploration. They didn’t just stumble into solutions — they went looking for them. That’s a non-trivial distinction, and the paper’s experiments show a clear performance gap between agents that are merely adaptable and agents that have learned that curiosity is a reliable path to mastery.

Still, it’s early days. The maze environments are pretty abstract, and Krazy World is a bespoke benchmark. The leap from these controlled tests to a real robot trying not to break itself while exploring a kitchen is enormous. What’s genuinely encouraging is the shift in thinking: exploration isn’t being treated as a hack or an add-on but as a first-class citizen of the meta-learning process. If that idea scales, the next generation of few-shot learners might come pre-loaded with something that looks a lot like common sense curiosity.