Intermediate · Systems

Structured Output

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. Constraining an LLM's generation to a defined schema (JSON, regex, grammar) so downstream code can parse it reliably.

Technical Definition

Constraining an LLM's generation to a defined schema (JSON, regex, grammar) so downstream code can parse it reliably.

How it works

Implementations include OpenAI's JSON mode and json_schema response_format, Outlines, jsonformer, llama.cpp grammars, and SGLang's regex-guided decoding. They work by masking the token logits at each step so only tokens consistent with the schema are sampled, guaranteeing valid output without retries. Essential for tool calling, data extraction, and agentic pipelines.

Related Concepts

• Function Calling — Provider-defined schemas that let an LLM invoke developer-supplied functions in a structured way.
• Tool Use — The ability of an AI model, especially an LLM agent, to autonomously identify, select, and utilize external tools and APIs to accomplish tasks.