Structured Output Is Not a Guarantee
The invoice parses perfectly. Every field is there, the total is a number, the currency is one of the allowed values, the date is a date. Your pipeline writes it downstream without a murmur — and the total is wrong. The model transposed two digits, the schema never noticed, and nobody was ever going to notice, because the check everyone trusted had already passed.
This is the misunderstanding we see most often in production AI work right now. Structured output, schema-constrained generation, is a real advance, and teams keep reading its guarantee as bigger than it is. The guarantee covers the shape of the answer. It was never about the truth of the values inside it.
What the constraint actually buys
The mechanism matters here, because the mechanism is what makes the guarantee strong in one place and absent in another. When you send a JSON Schema with the request, the provider compiles that schema into a grammar and constrains generation against it: tokens that would break the schema are not available to the model as it writes. OpenAI describes Structured Outputs as ensuring the model will always adhere to your supplied JSON Schema, so a required key is not omitted and an enum value is not invented. Anthropic describes its implementation the same way: a guarantee delivered through constrained decoding.
That buys you something genuine. A whole class of plumbing failures disappears: no more prose wrapped around the JSON, no missing field three lines before a crash, no retry loop whose only job is begging for the same shape again. If your system consumes model output in code (and in production, something always does), this is the difference between a format you hope for and a format you can build on.
But notice what the grammar cannot see. It knows that total must be a number. It does not know what the invoice said. It knows status must be one of four values. It does not know which one the document in front of it supports. Shape is a property of the answer alone; truth is a relationship between the answer and the world, and the grammar only ever sees the answer.
Three ways a valid answer is still wrong
The first is the plain wrong value: a transposed total, a date read from the wrong line of the document, a name taken from the signature block instead of the letterhead. Every field is the right type. The extraction is simply incorrect, and schema validation is structurally incapable of catching it.
The second is internal inconsistency. Line items that sum to 4,280 and a total field that says 4,820. A start date after the end date. A headcount of twelve and a list of nine names. A schema describes fields one at a time; many production errors live in the relationships between fields, which is exactly the territory a schema does not cover.
The third is subtler, and it is created by the constraint itself. In OpenAI's strict mode, every property in the schema is required — there is no honest way to say "the document doesn't state this." So when the source is silent, the model still has to put something in the field. A required field pointed at a document that doesn't contain the answer manufactures a plausible value. The tighter you make the schema, the more important it becomes to give the model an explicit way to say "not stated" (a null where the schema allows one, or a dedicated value in the enum), and to treat that answer as a routing signal, not a failure.
The guarantee has edges, and the vendors say so
Read the provider documentation closely and the guarantee is narrower than the marketing version. Anthropic's documentation is unusually candid about this. Some constraints in your schema — numeric bounds, string lengths — are not enforced during generation at all; the SDKs rewrite them into field descriptions, and validating SDK helpers check them against the original schema afterwards. A refusal or a response cut off at the token limit can return output that doesn't match the schema. Even enum casing is not guaranteed: a value can come back differing from the schema only in capitalization, with no error raised.
None of this is a reason to avoid structured output. It is a reason to keep your own validator downstream of it, permanently, even on the days it never fires. The bad days are the ones that matter.
Validation first, repair second, rejection always available
The production pattern we argue for has three layers, and the order is the point.
First, deterministic semantic checks — the ones a schema can't express. Totals against line items. Values against the source text: does this figure actually appear in the document? Codes and identifiers against the systems that own them: does this purchase order exist? These checks are ordinary code, cheap to run, and they catch the errors that actually cost money.
Second, bounded repair. When a semantic check fails, send the output back once — with the specific error attached, not a generic "try again." "Total does not match the line-item sum of 4,280; re-read the source" is an instruction a model can act on. One attempt, occasionally two. A repair loop with no bound is not robustness; it is a system that has decided to keep guessing until something passes, and something eventually will.
Third, a reject path that is a real destination: a queue a person works, with the source, the failed output, and the failed check all attached. The reject queue is also your best design feedback. A field that lands there every week is telling you the schema, the prompt, or the source document is wrong, and that signal is worth more than another retry.
Where this leaves the schema
None of this argues against schemas. It argues for treating the schema as the first layer of a validation stack rather than as the stack. Design it deliberately: field order matters, because generation follows it — put the evidence a value rests on before the value where you can, and let the model work outward from the source instead of committing early and rationalizing backwards. Keep schemas inside the providers' complexity limits, and expect the first request against a new one to pay a grammar-compilation cost that later requests don't.
The honest summary is short. Structured output ended the era of parsing apologies: the almost-JSON, the missing key, the shape that changed between calls. What it did not end, and never claimed to if you read the documentation it ships with, is the need to check whether the answer is right. Shape is guaranteed. Truth is still your job.
Work with us
If your team's extraction or agent pipeline trusts schema-valid output a little more than it should — and most do, right up until the first wrong value reaches a system of record — that gap between shape and truth is exactly the kind of problem we work on. Tell us about it through Trida AI's intake.