Module 05 — Structured Output (JSON & Schemas)

Goal: Reliably get the model to return data in a strict, machine-readable shape (usually JSON) so other programs can use it. This is the bridge from “chatbot” to “software component.”

---

5.1 Why structured output matters

When an LLM’s output feeds another program — a database, a UI, an API call, a spreadsheet — you can’t have it reply with “Sure! Here are the results 😊 …”. The program needs predictable structure: the same fields, the same types, every time.

Structured output means constraining the model to produce data in a defined format, most commonly JSON (JavaScript Object Notation). Examples of when you need it:

• Extracting fields from text (name, email, amount, date).
• Classification with confidence scores.
• Generating records to insert into a database.
• Producing arguments for a tool/function call (Module 06).
• Any “the answer must be parseable” situation.

---

5.2 JSON in 90 seconds (for non-coders)

JSON is a simple text format for structured data. Two main containers:

• Object { } — a set of "key": value pairs (like a labeled form).
• Array [ ] — an ordered list of values.

Value types: string "hello", number 42 or 3.14, boolean true/false, null, nested object, or array.

{
  "name": "Maria Gomez",
  "age": 34,
  "is_member": true,
  "tags": ["vip", "newsletter"],
  "address": {
    "city": "Austin",
    "zip": "78701"
  }
}

Rules that trip people up:

• Keys and string values use double quotes ("), never single.
• No trailing comma after the last item.
• Commas separate items; colons separate key and value.

If any rule is broken, the JSON is “invalid” and a program can’t parse it. Your job is to get the model to emit valid JSON every time.

---

5.3 The basic recipe: describe the exact schema

Tell the model precisely which fields you want, their types, and to output only JSON.

Extract the following from the text and return ONLY a JSON object with these fields:
- "name": string, the person's full name
- "email": string, their email address, or null if none
- "intent": one of "complaint", "question", "praise"
- "urgent": boolean, true if they need a fast response

Return only the JSON, with no extra text, no markdown, no code fences.

Text: "Hi, this is Sam Patel (sam@acme.io). My order never arrived and I need
this resolved today!"

Expected output:

{"name": "Sam Patel", "email": "sam@acme.io", "intent": "complaint", "urgent": true}

Key moves in that prompt:

1. List every field with its type and meaning.
2. Constrain enums (“one of …”) so categories stay in your allowed set.
3. Handle missing data (“or null if none”) so the model doesn’t invent or drop fields.
4. “Return ONLY the JSON, no extra text/markdown” — stops the chatty “Here you go:” wrapper that breaks parsers.

---

5.4 Show the shape with an example (few-shot for JSON)

Combine with Module 04’s few-shot technique. One or two examples lock the format hard:

Convert each sentence into JSON like the examples.

Sentence: "Book a table for 4 at 7pm on Friday."
JSON: {"action": "reserve", "party_size": 4, "time": "19:00", "day": "Friday"}

Sentence: "Cancel my 2-person booking tomorrow."
JSON: {"action": "cancel", "party_size": 2, "time": null, "day": "tomorrow"}

Sentence: "Reserve for 6 people Saturday at noon."
JSON:

Examples eliminate ambiguity about field names, casing, and how to encode edge cases (like null).

---

5.5 Output priming: start the JSON for the model

As in Module 04, you can begin the answer to force structure. Provide the opening brace/field:

... (instructions) ...
JSON:
{

By ending your prompt with {, the model is strongly nudged to continue valid JSON. In APIs that support an assistant “prefill,” put { (or [) there. Combine with a stop sequence so generation halts cleanly.

---

5.6 The reliable way: native structured-output features

Hand-prompting JSON works but isn’t bulletproof — the model might add a stray comment or malform a field. Modern model APIs offer features that guarantee (or strongly enforce) valid structure. Learn these; they’re how pros do it in production:

• JSON mode: a setting that forces the output to be syntactically valid JSON. You still describe the fields in the prompt, but you’re guaranteed parseable JSON.
• Structured Outputs / JSON Schema: you supply a formal JSON Schema (a precise spec of fields, types, required-ness, allowed values), and the API constrains the model so the output must conform. This is the gold standard: no missing fields, no wrong types, no invented keys.
• Tool/function calling: you define a function with a typed parameter schema, and the model returns arguments matching it (Module 06). This is itself a structured-output mechanism.

A JSON Schema looks like:

{
  "type": "object",
  "properties": {
    "name":   { "type": "string" },
    "email":  { "type": ["string", "null"] },
    "intent": { "type": "string", "enum": ["complaint", "question", "praise"] },
    "urgent": { "type": "boolean" }
  },
  "required": ["name", "intent", "urgent"],
  "additionalProperties": false
}

"required" forces those fields to appear; "enum" restricts allowed values; "additionalProperties": false blocks extra invented keys. Feeding a schema like this to a structured-output API gives you near-perfect reliability. Check your specific provider’s docs for the exact parameter name — the concept is the same across providers.

Rule of thumb: For anything beyond a quick experiment, prefer a native JSON-mode / schema feature over hoping a plain prompt yields valid JSON.

---

5.7 Validate, then handle failures (defense in depth)

Even with good prompting, in real systems you should validate the output in code and have a fallback. Standard pattern:

1. Ask the model for JSON (ideally via schema/JSON mode).
2. Try to parse it (e.g., json.loads in Python).
3. Validate it against your schema (right fields, right types, allowed values).
4. If it fails:
     - Option A: re-prompt, including the error: "Your last output was invalid
       JSON because X. Return corrected JSON only."
     - Option B: retry the call.
     - Option C: fall back to a safe default and log it.

A tiny Python sketch:

import json

raw = call_model(prompt)          # the model's text output
try:
    data = json.loads(raw)        # parse
    assert "intent" in data       # minimal validation
except (json.JSONDecodeError, AssertionError):
    raw = call_model(repair_prompt(raw))   # ask it to fix, or retry
    data = json.loads(raw)

Libraries exist to make this clean (e.g., Pydantic in Python validates parsed JSON against a typed model and can auto-generate the JSON Schema for you). The principle: never blindly trust the string — parse and validate.

---

5.8 Common pitfalls and fixes

Pitfall	Symptom	Fix
Chatty wrapper	”Sure! Here’s the JSON: …”	Add “Return ONLY JSON, no other text.” Use JSON mode.
Markdown code fences	Output wrapped in json …	Tell it “no code fences,” or strip them before parsing.
Trailing commas / single quotes	Parse error	Use a native JSON-mode/schema feature; validate + repair.
Missing fields	Key absent	Mark fields required (schema) or “always include all fields, use null if unknown.”
Invented categories	Value outside your set	Constrain with enum / “must be one of …”.
Inconsistent key names/casing	Name vs name	Show exact keys via example/schema; be explicit.
Numbers as strings	"age": "34"	Specify types in the schema; “age must be a number, not a string.”
Hallucinated values	Made-up email/amount	”Use null if not present; do not guess.”
Extra keys	Unexpected fields	additionalProperties: false, or “only these fields.”

---

5.9 Beyond JSON: other structured formats

JSON is the default, but you may want:

• CSV / tables: great for spreadsheets. Specify exact columns and that the first row is headers. Watch out for commas/quotes inside values.
• Markdown tables: for human-readable structured output in chat.
• XML/tagged output: sometimes easier for the model to produce reliably and easy to extract with simple tag matching; useful for wrapping reasoning vs. answer (<answer>…</answer>).
• YAML: human-friendly but whitespace-sensitive; less robust to parse than JSON.

Pick the format your downstream consumer needs. For program-to-program, JSON (with a schema) is usually safest.

---

Exercises

1. Extractor. Write a prompt that takes a free-text customer message and returns JSON with name, email (or null), product, and sentiment (enum: positive/neutral/negative). Test on 5 varied messages, including ones missing a field.
2. Enum discipline. Add a priority field constrained to low/medium/high. Feed an ambiguous message and confirm it never invents a fourth value.
3. Repair loop (concept). Take a deliberately malformed JSON string and write a “fix this to valid JSON, output only JSON” prompt. Confirm it repairs it.
4. Schema thinking. Write a JSON Schema (by hand) for an event object with title (string, required), attendees (number), online (boolean), and tags (array of strings). Disallow extra properties.
5. Format swap. Take the extractor from #1 and ask for the same data as a Markdown table instead. Note what’s easier/harder.

---

Key takeaways

• Structured output (usually JSON) turns the model into a software component other code can rely on.
• In the prompt: list every field with type and meaning, constrain categories with enums, handle missing data explicitly, and say “return ONLY JSON.”
• Examples and output-priming strengthen format adherence.
• For production, prefer native JSON-mode / JSON-Schema / function-calling features — they enforce validity.
• Always parse and validate in code, with a repair-or-retry fallback. Never trust the raw string blindly.

Next: Module 06 — Tool Calling / Function Calling.