Cognitive

Part 4 · Generative and Multimodal Models

Reasoning Models

Reasoning models spend extra effort working through steps before they give a final answer.

3 min readBuilds on Reinforcement Learning, Instruction Tuning and RLHF

Reasoning models work through intermediate steps before answering.

Key concepts

  • Chain of thought breaks a problem into intermediate steps.
  • Reasoning models are trained to spend more computation on hard questions.
  • They can try different approaches, check work, and backtrack.
  • Extra thinking raises cost and latency, so it is not always worthwhile.
  • Visible reasoning is useful, but it may not be a complete record of how the model decided.

Reasoning models spend additional computation before committing to an answer.

Figure 22.1. Reasoning models spend additional computation before committing to an answer.
More reasoning raises cost faster than it improves accuracy past a certain point.

Formal view and evidence

  • Established. Chain-of-thought prompting improves performance on many multi-step benchmarks. Reinforcement learning with verifiable rewards, such as checking math answers or passing tests, can induce longer reasoning.
  • Debated. How much benchmark gains reflect general reasoning rather than task-specific patterns.
  • Open problem. faithfulness, meaning whether stated reasoning actually reflects the factors that determined the answer, and how far these gains transfer to open-ended tasks.

Common misconception

Common belief

"The reasoning text shows exactly what the model is thinking."

In practice

It can be useful evidence, but it may leave out what truly influenced the answer.

Key takeaways

  • Reasoning models spend more computation before answering.
  • Verifiable rewards can train longer problem-solving behavior.
  • Visible reasoning is not always a faithful record of the computation.