Frontend Beauty
ESC

Type to search the knowledge base.

Structured Outputs and JSON Schema

Force the model into a schema instead of parsing its vibes. When constrained decoding helps, and when the model still fills fields with junk.

intermediate6 min read
  • json
  • schema
  • zod

JSON.parse on a chat reply will fail on a Tuesday. A markdown fence, a preface (“Sure, here is the JSON”), a trailing comma, an em-dash in a string you did not expect. Structured outputs are the adult version: you hand the API a JSON Schema (or a Zod schema that compiles to one), and constrained decoding only allows tokens that keep the string valid against that schema.

Valid JSON can still be a wrong invoice. The schema says totalCents is an integer. It does not say the integer matches the line items. Parse, then apply business rules. If either step fails, do not show a success toast.

Key takeaways

  • Constrained decoding makes the shape reliable. It does not make the values true.
  • Prefer JSON Schema / Zod over “respond as JSON” in the prompt.
  • JSON mode (valid JSON, any keys) is weaker than a schema with additionalProperties: false.
  • Validate money, ids, and enums in your code after the object parses.

Structured outputs beat JSON.parse

Unconstrained chat is a string source. Frontends then grow a graveyard of repair functions: strip fences, find the first {, hope.

function parseJsonFromVibes(text: string): unknown {
  const fenced = text.match(/```(?:json)?\s*([\s\S]*?)```/);
  const raw = fenced?.[1] ?? text;
  return JSON.parse(raw.trim()); // throws on Tuesday
}

That helper is a smell. Use it only as a fallback for a model that cannot constrain, and still validate with Zod. For anything you ship, turn on structured outputs on the provider.

OpenAI’s structured outputs (JSON Schema in the response format, or function calling with a schema) are built for this. Anthropic documents structured outputs on supported models too. Vendor field names move. The idea does not: schema in, object out.

Schema in, object out

Define the object in Zod. That is the same habit as runtime validation with schemas: TypeScript types erase. The schema runs.

import { z } from 'zod';

const LineItem = z.object({
  sku: z.string().min(1),
  qty: z.number().int().positive(),
  unitCents: z.number().int().nonnegative(),
});

const InvoiceDraft = z.object({
  vendorName: z.string().min(1),
  invoiceDate: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
  currency: z.enum(['USD', 'EUR', 'GBP']),
  items: z.array(LineItem).min(1),
  totalCents: z.number().int().nonnegative(),
  notes: z.string().nullable(),
});

type InvoiceDraft = z.infer<typeof InvoiceDraft>;

function assertBusinessRules(draft: InvoiceDraft) {
  const sum = draft.items.reduce((acc, it) => acc + it.qty * it.unitCents, 0);
  if (sum !== draft.totalCents) {
    throw new Error(`total mismatch: items=${sum} total=${draft.totalCents}`);
  }
}

async function extractInvoice(text: string): Promise<InvoiceDraft> {
  const res = await fetch('/api/extract-invoice', {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ text }),
  });
  if (!res.ok) throw new Error(`extract ${res.status}`);
  const body: unknown = await res.json();
  const draft = InvoiceDraft.parse(body);
  assertBusinessRules(draft);
  return draft;
}

On the server, map InvoiceDraft to the provider’s JSON Schema. Strict mode typically wants additionalProperties: false and every property listed in required. Optional fields become unions with null, not omitted keys. That constraint is annoying and it is the point: the model cannot invent taxCode.

If you use the Vercel AI SDK, generateObject is the wrapper. The raw idea is the same. See AI SDK structured output when you are on that stack.

Constrained decoding vs JSON mode vs tools

Mechanism What it guarantees What it does not
Prompt only (“return JSON”) Nothing Syntax, keys, types
JSON mode / json_object A parseable object Key names, enums, extra fields
Structured outputs (schema) Object matches the schema That totalCents is correct
Tool / function schema The call matches the schema That your handler should run it

Use structured outputs when the assistant is the object (extract, classify, form-fill). Use a tool schema when the model is proposing a side effect your server will run. Function calling is structured output aimed at your runtime, not at the user’s screen.

Constrained decoding works by masking invalid next tokens as the JSON is built. That is why it beats regex. It also means a too-tight schema can make the model ramble in a string field it is allowed to fill, or refuse. Log refusals as a first-class status. OpenAI documents explicit refusal shapes on some APIs. Handle them. Do not parse a refusal string.

Valid JSON, wrong invoice

The schema said invoiceDate is YYYY-MM-DD. The model wrote 2026-02-31. Regex passed if you were sloppy. Calendar check fails. Put the calendar check in assertBusinessRules, not in the prompt.

Other junk that survives a schema:

  • Hallucinated SKUs that match z.string().min(1)
  • Enums picked because they are first in the list
  • notes stuffed with chain-of-thought you did not want persisted
  • Quantities that do not match the source PDF because vision misread a column

Grounding is separate: if the source text is missing a total, the model will still emit an integer unless you allow null and instruct it to refuse. Your prompt engineering (the spec) and the schema have to agree. See prompt engineering fundamentals.

Do not “repair” a failed business rule by asking the model to try again in an unbounded loop. Cap retries. Surface the mismatch to a human when money is involved.

Should you stream a structured object?

You can stream tokens of a JSON value. You cannot JSON.parse until the object closes. Partial JSON is a syntax error, not a draft object.

Options:

  1. Wait for the end, then parse. Simplest. Fine for small extracts.
  2. Use a streaming parser (or the SDK’s partial object helper) that yields a best-effort tree as keys close. UI can fill fields as they complete. Still run Zod at the end.
  3. Do not bind a Submit button to a partial. A missing totalCents is not 0.

If the stream aborts, throw away the partial or mark it incomplete. Writing a half object to the database is how you invent invoices.

Schema size is prompt. A 400-line JSON Schema is tokens you pay for, and it can hurt cache locality if it changes. Keep the object the UI needs, not your entire domain model.

Out loud, in an interview

“Structured outputs constrain decoding to a JSON Schema so I get a parseable object instead of fencing and regex. I still run Zod and business rules after that, because a valid total can be the wrong total. JSON mode is weaker: any keys. For side effects I use tool schemas and authorize in the handler. I would not JSON.parse a partial stream, and I would not retry repairs without a cap.”

Further reading (primary sources)

Related guides