Test-Time Compute
Spending more computation while a model is answering — generating longer reasoning, sampling multiple solutions, or searching over options — instead of only scaling the training run.
Test-time compute is the idea that a model can get better at a problem by thinking longer, not just by being trained bigger. Instead of a fixed forward pass, the system invests extra inference compute in the answer.
Ways to spend it:
- Longer chains of thought: let the model reason step by step before answering.
- Self-consistency: sample many answers and pick the most common result.
- Best-of-N with a verifier: generate several candidates and score them, often with a reward model or unit tests.
- Search: explore a tree of partial solutions, as in tree-of-thought methods.
The approach shifts the cost curve: training is paid once, inference is paid on every query, so test-time scaling is most attractive for hard, high-value problems. Research suggests returns depend on the task — problems with verifiable answers benefit most, while open-ended tasks plateau. A related trend is adaptive compute, where a router decides per request how much thinking a question deserves, keeping easy queries fast and cheap.