FollowAgents AI AGENTS DIRECTORY
Home / AI Agent Glossary / Test-Time Compute
Ecosystem & Emerging Terms

Test-Time Compute

Also called: inference-time compute · test-time scaling · inference-time scaling

Test-time compute is computation spent while a model answers a query — rather than during training — used to improve the result through longer chains of reasoning, multiple sampled attempts, search over options, or self-checking.

QuestionReasoning path AReasoning path BPick best answer
One way to spend test-time compute: explore several reasoning paths for the same question in parallel, then select or vote on the best result.

For years, the main lever on model capability was training: bigger models, more data, more training compute. Test-time compute is a second lever that became prominent with reasoning-focused models: hold the trained model fixed, and spend more computation at the moment it answers a hard question.

That spending takes a few forms. The model can produce a longer chain-of-thought before committing to an answer. It can sample several independent attempts and then vote or pick the best. It can run a search — generate candidate steps, evaluate them, expand the promising ones. It can critique and revise its own draft (reflection-self-correction). All of these trade latency and token cost for a better chance of a correct answer, and all of them can be dialed up for hard problems and down for easy ones. Some models now expose an explicit "effort" or reasoning-budget setting that does exactly this.

For agents the relevance is direct. A planning step or a tricky tool-use decision benefits from more inference-time compute; a routine step doesn't need it. Industry framing in 2026 often describes delivered capability as the product of model quality, test-time compute, and tooling — with the latter two now worth optimizing as much as the model itself.

How it works

The common strategies: (1) longer reasoning — let the model generate more intermediate tokens before answering; (2) parallel sampling with selection — draw N answers at some temperature and aggregate by majority vote or a scoring model ("best-of-N"); (3) search — treat reasoning as a tree, score partial paths, and expand the best (beam search, MCTS-style approaches); (4) iterative refinement — draft, critique, revise for a few rounds. A controller decides how much to spend, sometimes from a user-set budget, sometimes adaptively based on the model's own uncertainty. More compute helps most on problems with verifiable or checkable structure (math, code, multi-step logic) and less on open-ended generation.

Example

An agent hits a subtask: "work out the correct order of database migrations given these 12 files and their dependencies." On low effort it answers directly and sometimes gets the order wrong. On high effort it generates several independent orderings, checks each against the stated dependencies, discards the ones that violate a constraint, and returns the one that survives — more tokens and a few seconds slower, but reliably correct.

How it differs

Test-time compute vs. training-time compute: training-time compute changes the model's weights once and benefits every future query. Test-time compute is spent per query, doesn't change the model, and can be adjusted per request — more for a hard problem, less for a simple one. They are complementary levers, not substitutes.

Common misconceptions

Often assumed: More test-time compute always produces a better answer.
Actually: It raises the odds of a correct answer mainly on problems with checkable structure; on open-ended tasks the gains flatten quickly, and it always costs latency and tokens.
Often assumed: A reasoning model's longer output means it's just being verbose.
Actually: The extra intermediate tokens are the mechanism — spending them on a chain of reasoning or self-checking is what raises accuracy, not filler.

FAQ

What is test-time compute?
Computation spent while a model answers — longer reasoning, multiple sampled attempts, search, or self-checking — used to improve the result without retraining the model.
What is the difference between test-time and training-time compute?
Training-time compute changes the model's weights once for all future queries; test-time compute is spent per query, leaves the model unchanged, and can be turned up or down per request.
Does test-time compute help every task?
It helps most on problems with verifiable structure like math, code, and multi-step logic; for open-ended generation the benefit is smaller and it still adds latency and cost.

Last checked: 2026-08-30

Related terms

Chain-of-Thought (CoT)

Chain-of-thought is a prompting technique where a model is asked to write out its intermediate reasoning steps before giving a final answer, rather than producing the answer directly.

Reflection / Self-Correction

Reflection (or self-correction) is when an agent evaluates its own output or action against the goal and revises it, instead of treating its first attempt as final.

Planning

Planning is the step where an agent turns a goal into an ordered set of smaller steps before — or while — it starts acting, so it isn't deciding what to do purely one action at a time with no larger structure.