Structured Output Is a Constraint, Not a Prompt

Asking a model to reply in JSON and hoping is not structured output. It is a coin flip that fails in production the day a stray sentence or a trailing comma breaks your parser.

Structure is a constraint you enforce, not a request you make. The reliable path is to bind generation to a schema through constrained decoding, grammars, or function-calling that only permits tokens the schema allows. The model cannot emit invalid output because invalid tokens are never on the table. You get a guarantee, not a probability.

This changes what you build on top. When output is guaranteed to parse, you can wire the model into real systems, calling a tool, writing a row, triggering a step, without a defensive tangle of regex and retries. The failure mode shifts from unparseable to merely wrong, which is a far better problem: wrong you can validate and catch; unparseable you cannot even inspect.

Prompting for a shape gets you a shape most of the time. Most of the time is exactly what does not survive contact with scale. If downstream code depends on the structure, make the structure impossible to violate, and spend your prompt budget on the content instead.