Tree of Thought: When Chain-of-Thought Isn't Enough
By Learnia Team
Tree of Thought: When Chain-of-Thought Isn't Enough
This article is written in English. Our training modules are available in French.
Chain-of-Thought follows a single reasoning path. But some problems require exploring multiple possibilities, backtracking, and comparing alternatives. That's where Tree of Thought comes in.
What Is Tree of Thought?
Tree of Thought (ToT) is a prompting technique where the AI:
- →Generates multiple possible next steps
- →Evaluates which paths are promising
- →Explores the best options further
- →Backtracks if a path fails
It mimics how humans solve complex puzzles—considering alternatives, not just following one line of thinking.
Chain-of-Thought vs Tree of Thought
Chain-of-Thought (Linear)
Start → Step 1 → Step 2 → Step 3 → Answer
One path, no alternatives considered.
If Step 2 is wrong, everything after fails.
Tree of Thought (Branching)
Start
├── Option A
│ ├── A1 ✓ (promising)
│ └── A2 ✗ (dead end)
├── Option B
│ ├── B1 → B1a → Solution! ✓
│ └── B2 ✗ (dead end)
└── Option C ✗ (pruned early)
Multiple paths explored. Dead ends abandoned. Best path found.
When Tree of Thought Helps
Puzzles and Games
Problem: "24 Game" - make 24 from [4, 5, 6, 3] using +, -, ×, ÷
CoT approach: Try one combination, hope it works
ToT approach: Systematically explore combinations, evaluate each
Planning Problems
Problem: "Plan a 7-day Europe trip hitting 5 cities efficiently"
CoT: Generate one itinerary
ToT: Generate multiple routes, compare travel times, optimize
Creative Problem Solving
Problem: "Design a mobile app for elderly users"
CoT: One design idea
ToT: Multiple concepts, evaluate usability of each, combine best elements
Search Problems
Problem: Find the best marketing strategy from 20 options
CoT: Analyze sequentially, pick first "good enough"
ToT: Evaluate multiple strategies, compare, pick optimal
The ToT Process
Step 1: Decompose
Break the problem into steps:
Problem: "Write a creative story with a twist ending"
Decomposition:
1. Choose a genre/setting
2. Establish characters
3. Build rising tension
4. Create the twist
5. Resolve the story
Step 2: Generate Options
At each step, brainstorm multiple possibilities:
Step 1 - Genre options:
A) Mystery in a small town
B) Sci-fi on a space station
C) Romance in 1920s Paris
Step 3: Evaluate
Assess each option's promise:
A) Mystery: ★★★☆☆ (common, but flexible for twists)
B) Sci-fi: ★★★★☆ (great twist potential, visual)
C) Romance: ★★☆☆☆ (harder to do unexpected twist)
Step 4: Explore Best Paths
Continue with promising options:
→ Pursue B) Sci-fi
Step 2 - Character options:
B1) Solo astronaut
B2) Ship crew
B3) AI companion
Best: B3 (AI companion opens twist possibilities)
Step 5: Backtrack If Needed
If a path hits a dead end:
B3 → twist idea 1: predictable ✗
B3 → twist idea 2: doesn't fit ✗
Backtrack to Step 1, try A) Mystery instead
Why ToT Works Better for Complex Problems
1. Avoids Early Commitment
CoT locks in decisions:
"The detective is named John..."
→ Stuck with this choice even if it creates problems later
ToT keeps options open:
Consider: John (detective), Sarah (journalist), Alex (suspect)
→ Choose based on what works best for the story
2. Enables Comparison
Strategy A produces: $50K revenue estimate
Strategy B produces: $75K revenue estimate
Strategy C produces: $60K revenue estimate
→ Choose B (can only compare with multiple paths)
3. Allows Recovery from Mistakes
Path going wrong? Backtrack.
CoT: Stuck with bad decisions
ToT: Return to last good state, try different branch
Real-World Example: Game of 24
Problem: Use 4, 9, 10, 13 to make 24 (each number once, any operations)
CoT Attempt
Let me try: 4 × 9 = 36... 36 - 10 = 26...
Can't use 13 to get to 24. Failed.
Try again: 10 + 13 = 23... 23 + 4 = 27...
Can't use 9 to get to 24. Failed.
Random attempts, might not find solution.
ToT Approach
Generate possible first operations:
- 4 + 9 = 13 (duplicate with existing 13, interesting)
- 4 × 9 = 36 (close to 24)
- 10 - 4 = 6 (small number, useful for multiplication)
- 13 - 9 = 4 (duplicate with existing 4)
Evaluate most promising: 10 - 4 = 6
With 6, 9, 13:
- 6 × 9 = 54... minus 13 = 41 ✗
- 13 - 9 = 4, 4 × 6 = 24 ✓
Solution: (13 - 9) × (10 - 4) = 24
Systematic exploration finds the answer.
ToT Performance (Research)
Yao et al. (2023) compared techniques on puzzle-solving:
| Technique | Game of 24 | Creative Writing | Planning | |-----------|------------|------------------|----------| | Standard prompting | 7% | 6/10 | 35% | | Chain-of-Thought | 4% | 6.5/10 | 42% | | Tree of Thought | 74% | 7.5/10 | 71% |
For search-like problems, ToT dramatically outperforms.
When NOT to Use ToT
Simple Questions
"What's the capital of Japan?"
→ Just answer directly. No tree needed.
Linear Problems
"Summarize this document"
→ CoT is sufficient. No branching helps.
When Speed Matters
ToT requires multiple evaluations and comparisons.
For real-time chat, it's too slow.
Key Takeaways
- →Tree of Thought explores multiple reasoning paths
- →Uses generate → evaluate → explore → backtrack cycle
- →Best for puzzles, planning, and search problems
- →Dramatically outperforms CoT on complex tasks (74% vs 4% on Game of 24)
- →Trade-off: More powerful but slower and costlier
Ready to Master Advanced Reasoning?
This article covered the what and why of Tree of Thought. But implementing these techniques effectively requires deep understanding and practice.
In our Module 3 — Advanced Reasoning Techniques, you'll learn:
- →Chain-of-Thought fundamentals
- →Self-Consistency for reliability
- →Tree of Thought implementation patterns
- →When to use each technique
- →Practical exercises with complex problems
Module 3 — Chain-of-Thought & Reasoning
Master advanced reasoning techniques and Self-Consistency methods.