02: Structured Output

The travel assistant can now remember the current conversation. Its answer is still prose.

That is fine for a human reader and annoying for code. A UI may want this shape:

{
  "city": "Hangzhou",
  "morning": "West Lake",
  "afternoon": "China National Tea Museum",
  "evening": "Hefang Street",
  "packing": ["umbrella", "light jacket", "comfortable shoes"]
}

If the model writes a list today, a paragraph tomorrow, and forgets packing the day after, the frontend and the next Python function both suffer.

This chapter adds structured output: the model must return data that code can check.

Code First

from __future__ import annotations

from pathlib import Path
from typing import Any

from agent_patterns_lab.runtime import Message, MockLLM, SchemaValidationError, Tracer, structured_complete


def main() -> None:
    tracer = Tracer()

    model = MockLLM(
        [
            '{"city":"Hangzhou","items":["West Lake","Tea Museum"]}',
            '{"city":"Hangzhou","morning":"West Lake","afternoon":"China National Tea Museum","evening":"Hefang Street","packing":["umbrella","light jacket","comfortable shoes"]}',
        ]
    )

    messages = [
        Message(
            role="system",
            content=(
                "Return ONLY JSON with keys: city, morning, afternoon, evening, packing. "
                "packing must be a list of strings."
            ),
        ),
        Message(role="user", content="Create a one-day Hangzhou itinerary."),
    ]

    def parse_itinerary(value: Any) -> dict[str, Any]:
        if not isinstance(value, dict):
            raise SchemaValidationError("expected a JSON object")
        required = ["city", "morning", "afternoon", "evening", "packing"]
        for key in required:
            if key not in value:
                raise SchemaValidationError(f'missing key "{key}"')
        if not isinstance(value["packing"], list) or not all(isinstance(x, str) for x in value["packing"]):
            raise SchemaValidationError('"packing" must be a list of strings')
        return value

    itinerary = structured_complete(
        model,
        messages,
        parser=parse_itinerary,
        schema_hint='{"city":"...","morning":"...","afternoon":"...","evening":"...","packing":["..."]}',
        tracer=tracer,
    )

    print(itinerary)
    trace_path = tracer.export_jsonl(Path(".traces") / "10_structured_output.jsonl")
    print(f"[trace] {trace_path}")


if __name__ == "__main__":
    main()

Run:

uv run python examples/10_structured_output.py

The final value is a Python dict, and the trace shows a repair attempt.

Let The Model Fail Once

The first MockLLM response is:

{"city":"Hangzhou","items":["West Lake","Tea Museum"]}

That is not the shape we asked for. It is missing morning, afternoon, evening, and packing.

This happens with real models too. Even when the system prompt says "return only JSON", the model may skip fields, use the wrong type, or add a sentence around the JSON.

The New Boundary: Parser

Do not judge the output by vibes. Write a parser:

def parse_itinerary(value: Any) -> dict[str, Any]:
    if not isinstance(value, dict):
        raise SchemaValidationError("expected a JSON object")
    required = ["city", "morning", "afternoon", "evening", "packing"]
    for key in required:
        if key not in value:
            raise SchemaValidationError(f'missing key "{key}"')
    if not isinstance(value["packing"], list) or not all(isinstance(x, str) for x in value["packing"]):
        raise SchemaValidationError('"packing" must be a list of strings')
    return value

That plain function changes the responsibility split:

Who	Owns
Model	Attempts to generate JSON matching the schema
Python parser	Checks fields and types
`structured_complete(...)`	Feeds validation errors back and retries

What `structured_complete(...)` Does

Think of it as a small loop:

flowchart TD
  P["Ask model for JSON"] --> O["Model output"]
  O --> E["Extract JSON"]
  E --> V{"Parser passes?"}
  V -->|Yes| R["Return Python object"]
  V -->|No| F["Append validation error"]
  F --> P

It does not make the model truthful. It turns format problems into detectable, retryable problems.

What It Fixes

Structured output helps when:

A frontend needs stable fields.
Another function needs to read a specific key.
Tests need to assert that packing is a list.
You do not want prose wrapped around JSON.

The travel assistant now returns a stable object instead of free-form text.

What It Does Not Fix

Structured output fixes shape, not facts.

The model can return valid JSON that is still bad advice:

{
  "city": "Hangzhou",
  "morning": "West Lake",
  "afternoon": "Outdoor tea walk",
  "evening": "Hefang Street",
  "packing": ["sunglasses"]
}

The shape is valid. If it rains tomorrow afternoon, the plan is still wrong.

Next: give the code access to external facts in 03: Tool Calling.