Paper link: https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf

This document investigates the strengths and limitations of Large Reasoning Models (LRMs), which are LLMs enhanced to generate detailed "thinking processes" before answering. The authors move beyond traditional performance benchmarks, which often suffer from data contamination, by using controllable puzzle environments that allow for precise manipulation of problem complexity. They found that while LRMs show advantages on moderately complex tasks, both standard LLMs and LRMs experience a complete accuracy collapse on highly complex problems, with LRMs exhibiting a surprising decrease in thinking effort as complexity increases beyond a certain point. Furthermore, analysis of the reasoning traces revealed inefficiencies and limitations in LRMs' ability to perform exact computation and reason consistently.

What are Large Reasoning Models (LRMs)?

Large Reasoning Models (LRMs) are a recent evolution of Large Language Models (LLMs) specifically designed to excel at reasoning tasks. They achieve this through explicit "thinking" mechanisms, such as generating detailed Chain-of-Thought (CoT) sequences and employing self-reflection before producing a final answer. Notable examples include OpenAI's o1/o3, DeepSeek-R1, Claude 3.7 Sonnet Thinking, and Gemini Thinking. These models have shown improved performance on reasoning benchmarks, suggesting a potential advancement in how AI systems handle complex problem-solving.

How are the capabilities of LRMs being evaluated?

Traditionally, LRM evaluation has focused on final answer accuracy on established mathematical and coding benchmarks. However, this approach has limitations, including potential data contamination and a lack of insight into the reasoning process itself. The provided source advocates for a new evaluation paradigm using controlled puzzle environments. These environments allow for precise manipulation of compositional complexity while maintaining consistent logical structures. This enables researchers to analyse not only the final answers but also the internal reasoning traces ("thoughts"), offering a more detailed understanding of how LRMs "think" and where their limitations lie.

What are the main findings regarding LRM performance as problem complexity increases?

The study reveals that frontier LRMs face a complete accuracy collapse beyond certain complexity thresholds, despite their advanced self-reflection mechanisms. When comparing LRMs to standard LLMs with equivalent computational resources, three distinct performance regimes are observed:

1. Low-complexity tasks: Standard LLMs surprisingly outperform LRMs and are more token-efficient.
2. Medium-complexity tasks: The additional "thinking" in LRMs provides an advantage.
3. High-complexity tasks: Both LRMs and standard LLMs experience complete performance collapse.

This indicates that while thinking mechanisms can help with moderately complex problems, they do not provide a fundamental solution to tackling highly complex reasoning challenges.

Do LRMs' reasoning efforts scale consistently with problem complexity?

Counterintuitively, the study finds that LRMs exhibit a scaling limit in their reasoning effort relative to problem complexity. Reasoning effort, measured by inference-time tokens used for thinking, initially increases with problem complexity up to a point. However, as problems approach the complexity threshold where accuracy collapses, LRMs begin to reduce their reasoning effort, even when they have an adequate token budget available. This suggests a fundamental limitation in the LRM's ability to leverage additional compute for thinking as problems become significantly harder.

What do the intermediate reasoning traces of LRMs reveal about their thinking process?

Analysis of the reasoning traces ("thoughts") provides insights into the LRM's problem-solving approach. The study observes complexity-dependent patterns:

• Simple problems: LRMs often find the correct solution early in their thinking but then inefficiently continue exploring incorrect alternatives, a phenomenon termed "overthinking".
• Moderate complexity: Correct solutions tend to appear later in the thinking process, after exploring various incorrect paths.
• High complexity: In the collapse regime, LRMs fail to generate any correct solutions within their thinking process.

This analysis highlights inefficiencies and scaling limitations in the self-correction mechanisms of current LRMs.

What limitations were observed in LRMs' ability to perform exact computations?

A surprising finding is the limitation of LRMs in performing exact computations and consistently following explicit algorithms. Experiments where the solution algorithm for the Tower of Hanoi puzzle was provided to the models showed no significant improvement in performance, and the accuracy collapse still occurred at similar complexity levels. This suggests that the limitation is not solely in devising a solution strategy but also in the ability to accurately execute logical steps and verify intermediate results, raising questions about their symbolic manipulation capabilities.

Do LRMs exhibit consistent reasoning across different types of puzzles?

The study observed inconsistent reasoning behaviour in LRMs across different puzzle types. For instance, the Claude 3.7 Sonnet thinking model could perform a significantly larger number of correct moves in the Tower of Hanoi puzzle (e.g., around 100 for N=10) before making an error, compared to the River Crossing puzzle where errors occurred much earlier (e.g., after only 4 valid moves for N=3). This suggests that LRM performance is highly dependent on the specific problem structure and potentially the presence of similar instances in their training data (data contamination), rather than possessing a generalized problem-solving capability.

Conclusion

The research challenges the current evaluation paradigm for LRMs and reveals fundamental limitations in their generalizable reasoning capabilities. The study highlights that while thinking mechanisms offer advantages at moderate complexity, they don't prevent collapse at high complexity. The counterintuitive reduction in thinking effort at high complexity and the struggles with executing explicit algorithms point to deep-seated limitations. The inconsistent performance across different puzzle types further suggests a reliance on learned patterns rather than true algorithmic reasoning. Key open questions for future research include better understanding LRMs' capacity for exact computation, improving their ability to follow and verify logical steps, and developing architectures that exhibit more consistent and generalizable reasoning across diverse problem structures.