Module 5 — Multi-Step Reasoning 🔄

Goal of this module: Learn how agents reason across multiple steps to solve hard problems — including the famous Chain-of-Thought and ReAct patterns. This module ties Modules 3 (planning) and 4 (tools) together into the engine that actually drives an agent.

5.1 What “multi-step reasoning” means

Some questions can be answered in one shot (“What’s the capital of France?”). Others require several connected thinking steps (“If I save $200/month and already have $1,500, how many months until I can afford a $4,000 trip, and what date is that?”).

Multi-step reasoning is the agent’s ability to chain several reasoning steps together — each building on the last — to reach a correct answer. It’s the difference between a snap guess and working through a problem carefully.

Analogy: Single-step = answering a flashcard. Multi-step = solving a word problem on paper, line by line.

5.2 Chain-of-Thought (CoT): thinking out loud

The simplest and most powerful reasoning technique is Chain-of-Thought (CoT): getting the model to show its reasoning step by step before giving the final answer.

Without CoT:

Q: A shop had 23 apples. They sold 17 and got 12 more. How many now?
A: 18   ← might be wrong; it just blurted a number

With CoT (just add “think step by step”):

Q: A shop had 23 apples. They sold 17 and got 12 more. How many now?
A: Let's think step by step.
   - Start: 23 apples
   - Sold 17: 23 - 17 = 6
   - Got 12 more: 6 + 12 = 18
   Final answer: 18 ✅

By forcing the model to lay out each step, it makes fewer mistakes — just like a student who shows their work. This costs more tokens but buys accuracy.

Why it works (intuition): Remember, the model predicts the next token from what’s already on the “desk.” If it writes out correct intermediate steps, each step gives it better context for the next one. Reasoning written down becomes scaffolding for the final answer.

5.3 The limitation of pure CoT

CoT is great for reasoning, but it happens entirely “in the model’s head.” It can’t:

Look up a fact it doesn’t know
Do exact math
Check the real world

That’s where we combine reasoning with action. Enter ReAct.

5.4 ReAct: Reason + Act (the most important agent pattern)

ReAct stands for Reason + Act. It’s the pattern where the agent alternates between thinking and using tools, over and over. You’ve actually already seen it — it’s the agent loop from Module 1, now with a name and structure.

The ReAct loop has three repeating parts:

THOUGHT     →  the agent reasons about what to do
ACTION      →  the agent uses a tool
OBSERVATION →  the agent reads the tool's result
   ↑                                    │
   └──────────── repeat ────────────────┘
   ...until it has the answer → FINAL ANSWER

Thought: “I need the current population of France.”
Action: web_search("population of France 2024")
Observation: “About 68.4 million.”
Thought: “Now I’ll double it using the calculator.”
Action: calculator("68400000 * 2")
Observation: “136,800,000”
Thought: “I have everything.”
Final Answer: “About 136.8 million.”

This interleaving of reasoning (CoT-style) and acting (tool use) is what makes ReAct so powerful: the agent can think, then check reality, then think again with new facts. It combines the brain of CoT with the hands of tool use.

5.5 A full ReAct trace (annotated)

Here’s what a ReAct agent’s output literally looks like. The labels (Thought/Action/Observation) are the structure that keeps it organized:

Question: Who is older, the current president of France or the UK prime minister,
          and by how many years?

Thought: I need both people's ages. First, find the French president's birth year.
Action: web_search("current president of France birth year")
Observation: Emmanuel Macron, born 1977.

Thought: Now the UK prime minister's birth year.
Action: web_search("current UK prime minister birth year")
Observation: [some name], born 1965.

Thought: 1965 is earlier, so the PM is older. Difference: 1977 - 1965 = 12.
Action: calculator("1977 - 1965")
Observation: 12

Thought: The PM is older by 12 years. I can answer now.
Final Answer: The UK prime minister is older than the French president by about 12 years.

Notice how each Thought sets up the next Action, and each Observation feeds the next Thought. That tight loop is the engine of a capable agent.

5.6 ReAct vs. plain Chain-of-Thought

	Chain-of-Thought (CoT)	ReAct
What it does	Reasons step by step	Reasons and uses tools
Can check facts?	❌ No (head only)	✅ Yes (via tools)
Can do exact math?	❌ Unreliable	✅ Via calculator tool
Best for	Pure logic/reasoning puzzles	Real-world tasks needing facts/actions

Rule of thumb: If a task needs only thinking, CoT is enough. If it needs information or actions from the world, use ReAct. Most real agents use ReAct.

5.7 Reflection: thinking about your thinking (preview)

A more advanced reasoning move is reflection — after producing an answer, the agent reviews its own work and fixes mistakes. (“Wait, let me double-check that calculation… actually I made an error.”)

We give this its own module (Module 7) because it’s so important for quality. For now, just know it’s an extra reasoning step layered on top of CoT/ReAct.

5.8 Self-consistency: ask, then vote

Another reasoning trick: self-consistency. Instead of reasoning once, the agent reasons through the problem several times (each may take a slightly different path), then picks the answer that comes up most often — like asking several experts and going with the majority.

Reason path 1 → answer: 18
Reason path 2 → answer: 18
Reason path 3 → answer: 17
                 ↓
        Majority vote → 18 ✅

✅ More reliable on tricky problems.
❌ Costs more (multiple runs). Use when accuracy really matters.

5.9 When reasoning goes wrong (and fixes)

Multi-step reasoning can fail in classic ways:

Failure	What happens	Fix
Looping	Agent repeats the same thought/action forever	Set a max-step limit; detect repeats
Going off-track	Reasoning drifts from the goal	Restate the goal in each step; re-plan
Bad observation handling	Ignores or misreads a tool result	Clear, structured tool outputs (Module 4)
Overthinking	12 steps for a 2-step task	Encourage “answer directly if you can”
Hallucinated actions	Calls a tool that doesn’t exist	Validate tool calls against the real tool list

Expert habit: Always cap the number of reasoning steps. An agent without a step limit can spin forever, burning money and time. A typical safety net is “stop after N steps and report what you have.”

5.10 Putting it together: the reasoning engine

Here’s how Modules 3, 4, and 5 fit:

PLAN (Module 3)  →  what's my overall approach?
       │
       ▼
REASON (this module, Module 5)  →  what's my next thought?
       │
       ▼
ACT with a TOOL (Module 4)  →  do it
       │
       ▼
OBSERVE  →  what happened?
       │
       └──── loop back to REASON, re-plan if needed ────┘

That combined loop — plan, then reason-act-observe repeatedly — is the working agent. Everything else (memory, reflection, multi-agent) makes this loop smarter.

5.11 Check yourself ✅

What does Chain-of-Thought do, and why does it improve accuracy?
What do the three repeating words in ReAct stand for / mean?
When should you use ReAct instead of plain CoT?
What is self-consistency?

Answers

It makes the model show its reasoning step by step before answering; writing out correct intermediate steps gives better context for each next step, reducing errors.
Thought (reason about what to do), Action (use a tool), Observation (read the result) — repeated until done.
When the task needs real-world information or actions (facts, math, APIs), not just internal logic.
Reasoning through a problem multiple times and taking the majority answer, to boost reliability.

5.12 Summary

Multi-step reasoning = chaining several thinking steps to solve harder problems.
Chain-of-Thought (CoT) = think step by step before answering → fewer mistakes.
ReAct (Reason + Act) = alternate Thought → Action → Observation to combine reasoning with tool use; this is the core engine of most agents.
Use CoT for pure reasoning, ReAct when you need facts or actions.
Self-consistency (vote across multiple runs) and reflection (Module 7) boost accuracy further.
Always cap steps to prevent loops.

Next up: Module 6 — Agent Memory. So far the agent forgets everything once the context window fills up. Let’s give it a real memory. 👉 06_Agent_Memory.md