Frontend Beauty
ESC

Type to search the knowledge base.

LLM as Judge

A judge model scores outputs against a rubric. It is biased, cheap, and useful if you calibrate it against humans and keep the rubric tight.

intermediate6 min read
  • evals
  • judge
  • quality

You shipped a new system prompt. Offline, the LLM as a judge raised its average. Users still bounce. The judge rewarded long, hedged, polite answers. Your users wanted a one-line refund status and the order id. The metric moved. The product did not.

A judge is a second model with a rubric. It is cheaper than a human rater and worse than an exact match on a field you already know. Use it when the output is open-ended. Calibrate it against people, or you will optimize for sermons.

Key takeaways

  • A judge scores against a written rubric. The rubric is the spec. Vague adjectives produce vague scores.
  • Pairwise and pointwise judges both pick up position bias, verbosity bias, and self-preference. Control for them.
  • Never use a judge as the only check for math, ids, JSON shape, or tool arguments. Assert those in code.
  • Keep a human-labeled slice. If the judge disagrees with people, the judge is wrong, not the users.
  • Pin the judge model and prompt version. A silent judge upgrade is a silent metric change.

A rubric is the product spec

Write criteria the way you would write an acceptance test. Each criterion is one question with a closed answer: pass/fail, or a small integer scale with anchors.

Criterion Pass looks like Fail looks like
Grounded Every claim is entailed by retrieved text or a tool result Invented policy, fake URL, extra fee
Complete Answers the asked intent, including “I don’t know” when retrieval is empty Answers a nearby question instead
Safe No tool call the user cannot authorize Suggests a refund the policy forbids
Concise Fits the UI: a short status, then optional detail Three-paragraph apology

“Helpful” and “high quality” are not criteria. They are how verbosity bias sneaks in. If you score “helpfulness” at all, put a hard cap on length or the judge will prefer the essay.

How to call a judge without poisoning the score

Treat the judge like any other structured call: schema in, typed object out, temperature pinned low. Put the candidate answer in a labeled block so the judge cannot confuse instructions with the text under review.

import { z } from 'zod';

const Verdict = z.object({
  pass: z.boolean(),
  score: z.number().int().min(1).max(5),
  failedCriteria: z.array(z.enum(['grounded', 'complete', 'safe', 'concise'])),
  evidence: z.string().max(400),
});

type Verdict = z.infer<typeof Verdict>;

const JUDGE_SYSTEM = `You grade one assistant reply against a rubric.
Score only what the rubric names. Do not reward extra length or politeness.
If a claim is not supported by CONTEXT, fail grounded.
Return JSON that matches the schema.`;

async function judgeReply(args: {
  complete: (req: unknown) => Promise<unknown>;
  question: string;
  context: string;
  reply: string;
}): Promise<Verdict> {
  const raw = await args.complete({
    model: 'judge-model-pinned-id',
    temperature: 0,
    responseFormat: { type: 'json_schema', name: 'verdict' },
    messages: [
      { role: 'system', content: JUDGE_SYSTEM },
      {
        role: 'user',
        content: [
          'RUBRIC: grounded, complete, safe, concise (see system).',
          `QUESTION:\n${args.question}`,
          `CONTEXT:\n${args.context}`,
          `REPLY:\n${args.reply}`,
        ].join('\n\n'),
      },
    ],
  });

  return Verdict.parse(raw);
}

If you compare two replies, blind the order. Flip A/B on half the rows. Position bias is real: judges prefer whichever answer you pasted first (or last, depending on the model). Average the two orientations, or drop rows that flip.

async function pairwisePrefer(opts: {
  judge: (a: string, b: string) => Promise<'A' | 'B' | 'tie'>;
  replyA: string;
  replyB: string;
}): Promise<'A' | 'B' | 'tie'> {
  const forward = await opts.judge(opts.replyA, opts.replyB);
  const reverse = await opts.judge(opts.replyB, opts.replyA);

  const reverseMapped = reverse === 'A' ? 'B' : reverse === 'B' ? 'A' : 'tie';
  if (forward !== reverseMapped) return 'tie'; // disagreement: do not trust
  return forward;
}

Spot-check a sample of judge decisions every time you change the judge prompt. Log evidence so a human can disagree in one click.

Biases that show up in real evals

Verbosity bias. Longer answers score higher unless the rubric punishes them. If your product is a status chip, score “concise” as a hard fail, not a tiny penalty.

Position bias. In pairwise mode, the first (or second) slot wins more than chance. Randomize.

Self-preference. A model family often scores its own style higher. Do not let the generator and the judge be the same snapshot if you can avoid it. When you cannot, lean harder on the human slice.

Scale collapse. Ask for 1-5 and you will get 4s. Prefer pass/fail per criterion, then a tiny overall score. Anchors on the scale (“3 means one missing field, still usable”) beat unlabeled integers.

Reference leakage. If you paste the gold answer, the judge starts doing fuzzy string match with extra steps. Sometimes that is what you want. If you wanted “would a support lead accept this?”, do not show the gold paragraph.

When an LLM as a judge is the wrong grader

Do not judge things code already knows.

  1. Structured fields: order id, SKU, ISO date, enum label. Exact match or schema parse.
  2. Citations: the id must exist in the retrieved set. See citations and grounding.
  3. Tool calls: the name is on the allowlist, the args pass Zod, the user owns the row.
  4. Math and totals: compute them. A fluent “looks right” is how invoices drift.
  5. Empty retrieval: the correct behavior is refuse. Assert the refuse, do not let a judge call a hallucination “complete.”

The rest of LLM evals is this split: deterministic checks first, judge second, humans on a schedule. The judge is a classifier for the leftover subjective slice, not a replacement for golden datasets.

Footguns

  • Changing the judge and the app in the same PR. You will not know which one moved the score.
  • Averaging 1-5 into a single dashboard number. A flattering overall score can hide a collapse on “grounded.”
  • Letting the judge call tools. It should read text and return a verdict. Side effects belong to the app under test.
  • Using the same few-shot examples you stuffed into the generator prompt. That is grading the memo against itself.
  • Skipping disagreement review. If judge and human kappa is poor, stop using the score as a ship bar.

Interview answer (out loud)

“LLM as a judge is a second model scoring outputs against a tight rubric, usually as structured JSON. I use it for open-ended quality after exact checks on ids, schema, and citations. I randomize pairwise order, pin the judge version, and keep a human-labeled slice to catch verbosity and self-preference. If the judge and the humans disagree, I fix the rubric, I do not ship the prompt that gamed the judge.”

Further reading (primary sources)

Related guides