Most large language models treat each token as an isolated event, linked only by static attention caches. A new technical report from Princeton researcher Yifan Zhang challenges this orthodoxy with the Recurrent Looped Transformer (RLT). The proposal is radical in its simplicity: it carries the decoder’s final hidden state and sliding-window attention cache directly into the processing of the next token. There is no reset at the boundary between prompt and response. This creates a continuous feedback loop where the model’s internal reasoning state evolves dynamically across the entire sequence.
The architecture pairs a causal encoder with a recurrent decoder. While the encoder processes tokens in parallel to build key-value memory, the decoder handles the heavy lifting of temporal continuity. In the reference configuration, Zhang specifies 48 encoder and 48 decoder layers. Each token effectively traverses 96 logical blocks, though the decoder adds cross-attention steps that increase computational load per block. This isn’t just copying activations; it’s parameter reuse. The system maintains a fixed per-token workload while allowing the structural depth of the reasoning path to grow linearly with the sequence length. After t tokens, the state path has traversed 48t decoder blocks.
Crucially, this is a design specification, not a benchmark victory lap. The report explicitly states there are no measured results for efficiency, reasoning quality, or scaling yet. It outlines a strict contract for reinforcement learning, requiring trainers to rebuild encoder memory and recurrent output from the sequence start before scoring actions. Old rollout states are never reused. This ensures the conditional distribution remains unchanged regardless of where the prompt-response split occurs. It’s a rigorous mathematical framework, but it lacks the empirical proof that usually drives industry adoption.
Zhang positions RLT against prior work like YOCO and DeepSeek-V4.1-Flash, which also leverage encoder-derived memory. However, RLT drops prompt-wide decoder skipping in favor of full temporal feedback. It builds on the Feedback Transformer but applies recurrence over the prompt itself. The real test will be whether hardware can keep up. The report admits that no exact parallel scan is assumed for the nonlinear decoder, meaning prefill speedups aren’t guaranteed. If the engineering hurdles of kernel fusion and checkpointing can be cleared, RLT could redefine how models maintain context. But until we see numbers, it remains a compelling hypothesis.
