Chain of Thought
Make AI show its work — and watch accuracy skyrocket.
What You'll Learn
- What chain-of-thought prompting is and why it works
- When to use step-by-step reasoning vs. direct answers
- How to trigger deep thinking with simple phrases
- Advanced: thinking budgets and structured reasoning
Why "Think Step by Step" Actually Works
When you ask AI a complex question directly, it tries to jump straight to the answer. Sometimes it nails it. Often it doesn't — especially with math, logic, or multi-part problems.
Chain-of-thought (CoT) prompting forces the model to reason through intermediate steps before reaching a conclusion. Research shows this can improve accuracy on complex tasks by 20-40%. It's not magic — it's giving the AI scratch paper.
Three Ways to Trigger Reasoning
1. The Simple Trigger
"Think step by step before answering."
Surprisingly effective. Just adding this phrase improves results on reasoning tasks.
2. The Structured Trigger
"First, identify the key variables. Then, analyze how they interact. Then, draw your conclusion. Show each step."
More control. You define the reasoning structure.
3. The Expert Trigger
"Approach this like a detective solving a case. Examine each piece of evidence, note what's relevant, eliminate what's not, then present your theory with supporting reasoning."
Combines persona with reasoning. The metaphor shapes how the AI thinks.
Not Everything Needs a Chain
Use CoT for: Math problems, logic puzzles, code debugging, strategic analysis, comparing options, anything where the reasoning matters as much as the answer.
Skip CoT for: Simple factual questions, creative writing, translation, formatting tasks. Forcing reasoning here just adds noise.
Thinking Budgets and Internal Reasoning
Some AI systems (including Claude) support extended thinking — where the model reasons internally before responding. You can guide this by specifying how much thinking you want.
Thinking Budget Prompt
"This is a complex architectural decision. Take your time. Consider at least 3 approaches, evaluate trade-offs for each, then recommend the best path with clear justification."
The key insight: you're not just asking for an answer. You're asking for a reasoning process. The quality of the process determines the quality of the output.
CoT for Code Debugging
Chain-of-thought shines when debugging. Instead of asking "why doesn't this work?", structure the AI's reasoning process.
Without CoT
"My React component re-renders infinitely. Here's the code: [code]. Fix it."
The AI might give you a fix that works — or it might guess wrong because it skipped the diagnosis.
With CoT
"My React component re-renders infinitely. Here's the code: [code].
Debug this step by step:
1. Identify all useEffect hooks and their dependency arrays
2. Trace which state changes trigger which effects
3. Find the circular dependency (state change → effect → state change)
4. Explain exactly which line creates the loop and why
5. Provide the minimal fix — change as few lines as possible"
The AI walks through each step, catches the real root cause, and provides a targeted fix instead of rewriting your component.
Self-Consistency: Multiple Reasoning Paths
One of the most powerful extensions of CoT is self-consistency — asking the AI to solve the same problem multiple ways and compare results.
Self-Consistency Prompt
"Solve this problem using three different approaches:
Approach 1: Work through it mathematically
Approach 2: Use an analogy or mental model
Approach 3: Reason from first principles
After all three, compare your answers. If they agree, you can be confident. If they disagree, identify where the reasoning diverges and determine which approach is most reliable for this type of problem."
This technique is especially valuable for ambiguous problems where there's no single "right" method. When multiple reasoning paths converge on the same answer, your confidence in that answer should be high.
CoT Anti-Patterns to Avoid
Over-specifying steps: If you dictate 15 micro-steps, you're doing the thinking for the AI. Give it 3-5 high-level steps and let it fill in the details. The sweet spot is structured enough to guide but flexible enough to reason.
Using CoT on trivial tasks: "Think step by step about what the capital of France is" adds latency and tokens without improving accuracy. Reserve CoT for tasks where reasoning genuinely matters. A good rule of thumb: if a human could answer the question in under 5 seconds, CoT probably won't help.
Ignoring the reasoning: If you ask for step-by-step reasoning but only read the final answer, you're missing the point. The reasoning is where you catch errors, learn the AI's assumptions, and refine your prompt.
Forgetting to ask for a conclusion: Some prompts trigger great reasoning but never ask for a clear final answer. Always end with "Based on this analysis, your recommendation is:" or "Therefore, the answer is:" to ensure a clear deliverable.
The Science Behind CoT
Chain-of-thought prompting was formalized in a 2022 paper by Google researchers (Wei et al.). The key finding: CoT improved performance on math word problems from 17.7% to 78.7% accuracy in PaLM 540B. The technique has since been validated across dozens of models and task types.
Why does it work? Large language models generate one token at a time. When you ask for a direct answer, the model must "compress" all reasoning into the first few tokens of its response. When you ask it to think step by step, each reasoning token becomes context for the next one — the model can build up to the answer incrementally.
This is why CoT works better on larger models. Smaller models may not have enough capacity to produce useful intermediate reasoning. If you're using a lightweight model and CoT isn't helping, it's not your prompt — the model may simply lack the reasoning capacity to benefit from the technique.
The practical takeaway: CoT is free. It costs a few extra output tokens but requires no special tools, no fine-tuning, no API changes. It's the single highest-ROI prompting technique available today.
An important nuance: newer models like Claude and GPT-4 have built-in "extended thinking" modes that perform internal CoT automatically. Even with these models, explicit CoT instructions help because they let you control the reasoning structure — deciding which steps the model takes rather than leaving it to default reasoning paths.
CoT for Decision Making
Chain-of-thought is uniquely powerful for decisions with multiple factors. Here's a decision-making template that produces consistently thoughtful analysis.
Decision Analysis Prompt
"I need to decide between [Option A] and [Option B]. Context: [situation].
Analyze this decision step by step:
1. List the top 3 criteria that matter most for this decision
2. Score each option against each criterion (1-10) with a one-sentence justification
3. Identify the biggest risk for each option
4. Consider: what would I regret more — choosing A and being wrong, or choosing B and being wrong?
5. Give your recommendation with confidence level (high/medium/low)"
Step 4 — the "regret minimization" question — is what makes this template exceptional. It forces the AI to consider asymmetric risks, not just balanced pros and cons. Many real decisions hinge on which downside is more painful, and this step surfaces that insight.
This template works for career decisions, technology choices, hiring decisions, and strategic planning. The structure stays the same — only the options and context change. Save it to your prompt library (Lesson 10) and reuse it whenever you face a complex decision.
Compare Direct vs. Chain-of-Thought
Pick a problem that requires reasoning. Ask it twice — once directly, once with chain-of-thought. Compare the quality and accuracy of both answers.
Direct: "What's the best database for my project?"
CoT: "I'm building [description]. Help me choose a database. First, identify my key requirements from the description. Then, list 3 database options that could work. For each, analyze pros and cons given my specific needs. Finally, recommend one with clear reasoning."