OpenAI Peels Back GPT‑4’s Layers, Finds 16 Million Interpretable Patterns

The black box problem isn’t solved, but OpenAI just cracked it open a little wider. The company has published new research detailing scalable methods that decompose GPT‑4’s internal representations into 16 million often-interpretable features. For years, the dense, unpredictable firing of neurons inside large language models has resisted easy explanation. A single activation seems to represent many concepts at once, making it impossible to reason about AI safety the same way an engineer specs out a car’s brakes. The new approach leans on sparse autoencoders, a technique designed to surface a small set of concepts relevant to any given input — more like how a human brain zeroes in on just a handful of ideas amid a flood of possible thoughts.

Scaling these autoencoders to frontier models has been the sticking point. Past work hit a wall; the sheer volume of concepts packed into models like GPT‑4 demanded a correspondingly massive number of features, and training them efficiently was an open challenge. OpenAI’s team claims to have cracked that scaling problem, introducing new methodologies that deliver smooth, predictable improvements as they dial up the feature count. The result is a 16-million-feature autoencoder applied to GPT‑4, alongside smaller versions tested on GPT‑2. The researchers also rolled out fresh metrics for evaluating whether those features actually mean anything.

So what do 16 million features look like? Some snap into focus with eerie clarity. One feature lights up for “phrases relating to things (especially humans) being flawed,” while others track concepts like price increases or rhetorical questions. The team has released a paper, code, and interactive visualizations so the broader research community can poke at the results themselves. It’s the kind of open‑ended sharing that suggests they know how preliminary this all is.

And the limitations are bluntly stated. Many features remain garbled, activating with no clear pattern or throwing off spurious signals. Passing GPT‑4’s activations through the autoencoder degrades performance to the level of a model trained with roughly 10x less compute. Full coverage might demand billions or trillions of features — a computational headache even with the improved techniques. Plus, finding features at a single point in the network is a far cry from understanding how the model computes them or uses them downstream. Still, OpenAI says the short‑term play is practical: use these features to monitor and steer model behavior in upcoming frontier systems. Trust through transparency is the long game, but for now, a partial map is a lot better than no map at all.