Module 10 — Generation & Prompting

We’ve found the best chunks. The final step is assembling them into a prompt and getting a grounded, trustworthy answer out of the model. This is the smallest stage — but a few details here separate a reliable system from one that quietly hallucinates.

1. Assembling the prompt

You construct a prompt with three ingredients:

Instructions — telling the model how to behave (ground itself, cite, admit ignorance).
Context — the retrieved chunks (from Module 09), clearly labeled.
The question — the user’s actual query.

A solid template:

You are a helpful assistant. Answer the user's question using ONLY the
context below. If the answer is not in the context, say you don't know.
Cite the source of each fact using its [Source] label.

Context:
[Source: Returns Policy, p.3] Laptops may be returned within 30 days...
[Source: Returns Policy, p.4] Refunds are issued to the original...
[Source: Shipping FAQ] Return shipping is free for defective items...

Question: How long do I have to return a laptop, and who pays shipping?

The model reads this and answers from the supplied text — not its blurry memory.

2. Grounding — the most important instruction

Grounding means basing the answer strictly on the provided context. The instruction “Answer using only the context; if it’s not there, say you don’t know” is the single biggest hallucination reducer in all of RAG. Without it, the model happily fills gaps from its frozen parametric memory (Module 02) — the exact behavior we built RAG to avoid.

A grounded system would rather abstain (“I don’t have that information”) than invent. For most business uses, “I don’t know” is far safer than a confident wrong answer. Always give the model an explicit permission-to-not-know path.

3. Citations — make every claim checkable

Ask the model to cite which chunk each fact came from (using the [Source] labels you put in the context). Two payoffs:

Trust: users can verify, which matters enormously in legal, medical, financial, and support settings.
Debuggability: when an answer is wrong, citations show you which chunk misled it — was it bad retrieval or bad reading? (This ties directly into evaluation, Module 12.)

Because each chunk carries metadata (Module 07), you can cite precisely: “Returns Policy, page 3.”

4. “Lost in the middle” — where you place chunks matters

A real, well-documented quirk: models pay most attention to the beginning and end of a long context, and can overlook material buried in the middle. This is nicknamed “lost in the middle.”

Practical consequence: put your most relevant chunks at the start and end of the context, not buried in the middle. Since you reranked in Module 09, you know which chunks are best — place them where the model will actually notice them. (This is also another reason not to over-stuff context with low-value chunks: they push the good stuff into the inattentive middle.)

5. Formatting the context clearly

Help the model tell chunks apart and attribute correctly:

Use clear delimiters/labels between chunks (like the [Source: ...] headers above).
Keep each chunk’s source label attached so citation is easy.
Don’t blur multiple chunks into one wall of text.

Small formatting care measurably improves both accuracy and citation quality.

6. The token budget

Remember the context window is finite (Module 02). Your prompt must fit: instructions + all the chunks + the question + room for the answer. If you cram in too many chunks, you might leave no space for a full answer, or trigger truncation. This is one more reason the “feed few, well-chosen chunks” discipline from Module 09 matters — quality and budget both favor a small, sharp context.

7. Handling the “no good context” case

What if retrieval finds nothing relevant? Don’t force an answer. Options:

Have the model say it doesn’t have the information (the grounding instruction handles this).
Optionally detect low retrieval scores and short-circuit to a graceful “I couldn’t find anything on that” before even calling the model.

A system that knows when to stay silent is more trustworthy than one that always produces something.

8. The full picture, end to end

You’ve now seen every stage. Here’s the whole journey of one question:

Question
  → (rewrite) → filter by metadata → hybrid search (top 50)
  → rerank (keep best 5) → assemble prompt with grounding + citations
  → model reads context, places best chunks at edges
  → grounded, cited answer  (or an honest "I don't know")

Everything in Modules 07–10 serves one goal: get the right text in front of the model and make it answer only from that text.

Check yourself

What is grounding, and why is the “only use the context / say you don’t know” instruction so powerful?
Give two reasons to require citations.
What is “lost in the middle,” and how do you work around it?
Why is forcing an answer when retrieval finds nothing a bad idea?

Next: Module 11 — Advanced Patterns →