Planning, Test-Time Compute & Deliberate Reasoning Scaling

Planning, Test-Time Compute & Deliberate Reasoning Scaling

What it is

Traditionally, LLMs generate responses using "System 1" thinking, fast, intuitive, next-token prediction. Deliberate reasoning scaling shifts this to "System 2" thinking, where the model is given extra computational resources at the moment of inference (test-time) to "think" before it speaks. Instead of a single pass, the model explores multiple reasoning paths, verifies its own steps, and corrects errors before delivering a final answer.

Why it matters

For complex agentic tasks, like software engineering or mathematical proofing, a single forward pass is often insufficient. By allocating more compute at test-time, we can improve model performance without needing to retrain the entire network. This allows AI systems to handle high-stakes tasks that require rigorous verification and strategic planning rather than just pattern matching.

Core concepts to master

• Test-Time Compute: The practice of increasing the amount of processing (tokens, search iterations, or verification loops) used to solve a specific prompt.
• Process Reward Models (PRMs): Unlike Outcome Reward Models that only grade the final answer, PRMs provide feedback on every individual step of a reasoning chain, allowing the system to spot exactly where a logic error occurred.
• Monte Carlo Tree Search (MCTS): A search algorithm that allows the model to simulate various reasoning paths, evaluate their potential for success, and backtrack to explore more promising alternatives.
• Thinking Tokens: The allocation of internal "hidden" tokens where the model performs scratchpad reasoning before producing the visible output.

Common mistakes

• Over-allocating compute: Spending massive resources on simple queries where a fast response is sufficient.
• Ignoring verification: Using MCTS or search without a reliable PRM to guide the search, leading the model to confidently explore incorrect paths.
• Confusing scaling laws: Assuming that more training data is the only way to increase intelligence, while ignoring the potential of inference-time scaling.

Connection to the track

This concept is the engine for Agentic Systems. While basic agents follow linear scripts, deliberative agents use these scaling techniques to plan complex multi-step goals, self-correct in real-time, and optimize their behavior through iterative search and verification.