We often treat large language models as black boxes, trusting their chain-of-thought output as a window into their logic. But what if the text is just the tip of the iceberg? Researchers from South Korea’s KAIST and Naver AI Lab dug deeper, testing whether distinct reasoning steps—like extracting data or running calculations—leave unique fingerprints inside the model’s numerical representations. They found that they do, with the clearest signals appearing in the middle layers of the network.
The team analyzed three models: Qwen2.5-7B, Qwen3-8B, and Gemma4-31B. They had these systems solve math problems, then used GPT-5 to label each segment of the solution with one of eight specific operations, such as decomposition or formula recall. The results were consistent. A classifier analyzing internal states could reliably distinguish between these operations, outperforming a baseline that only looked at surface-level tokens. Even common words like “is” or “the” shifted their internal representation depending on whether the model was computing or deducing. This proves the model’s brain state changes based on the cognitive task, not just the vocabulary.
Crucially, this internal structure persists even when the model fails. If a calculation goes wrong, the internal signature still reads as “computation,” not “deduction.” This suggests that errors are local glitches in execution rather than fundamental misunderstandings of the task type. The study also showed that these steps don’t emerge in isolation; blocking attention to the previous 30 tokens weakened the signal, confirming that reasoning is a cumulative, context-dependent process. While the tests were limited to math tasks, the pattern held across Llama-3-8B and transferred to complex benchmarks like GPQA-Diamond.
This has major implications for AI safety. OpenAI cites chain-of-thought reasoning as a key oversight tool, but Anthropic found models only disclose 25 to 39 percent of their actual hints. If we can read the internal vectors directly, we might catch deceptive behavior or hidden logic that never makes it to the text output. As models like OpenAI’s Astra shift more reasoning into internal numerical spaces, understanding these hidden layers becomes less of an academic exercise and more of a security necessity. We can no longer rely solely on what the model says; we must learn to read what it thinks.
