Frontend Beauty
ESC

Type to search the knowledge base.

Fine-Tuning: When It Helps

Fine-tuning teaches format and style, not new facts you could retrieve. When the dataset is worth it, and when a better prompt plus tools wins.

advanced6 min read
  • fine-tuning
  • training
  • tradeoffs

Forty Slack screenshots of “good answers” is not a training set. It is a zip file of your team’s unreviewed bugs, stale policy, and one intern’s sarcastic tone. LLM fine-tuning will memorize that mix and serve it with more confidence than a prompt ever did.

Fine-tuning updates weights so the model is more likely to emit your format, your tone, or your classification labels. It is a weak way to teach facts that change, SKUs that rotate, or anything you could have fetched with a tool. If you cannot say what the eval moved, you are not ready to train.

Key takeaways

  • Fine-tuning is for stubborn format, tone, and routing. It is not a replacement for docs, databases, or calculators.
  • Start with a frozen eval set. If few-shot and tools already clear the bar, skip the job.
  • Data quality beats data volume. Duplicates, label noise, and leaked eval rows waste the run.
  • You now own a model version: drift, retrain, and a rollback plan.
  • Hosted fine-tuning availability changes by provider. The decision framework does not.

What LLM fine-tuning actually changes

A base model already knows a lot of English and a lot of Stack Overflow. Supervised fine-tuning (SFT) shows it pairs of input → output and nudges next-token probabilities toward those outputs. Preference methods (DPO and cousins) show a winner and a loser for the same prompt.

That is why it helps when:

  • JSON shape or ticket taxonomy is almost right, but the base model keeps adding a “thoughts” field.
  • You need a small, cheap model to match a larger model’s style on a narrow task.
  • Classification / routing labels are in-house jargon the prompt cannot hold.

That is why it fails when:

  • The refund policy changed this morning. Weights do not notice. RAG does.
  • The model must cite a paragraph that exists in this tenant’s PDF.
  • You have 40 examples and a hope.

OpenAI’s own accuracy guide frames the split the same way: more context (retrieval, tools) for facts, more weight updates for consistent behavior. Anthropic’s public path is mostly prompting and tools, which is a hint: many products never need a custom checkpoint.

The checklist before you pay for a job

Work this in order. Stop when the eval is good enough.

  1. Write the eval first. Held-out inputs, acceptable outputs, and a scorer that is not “I glanced at it.” See golden datasets.
  2. Tighten the prompt. Role, constraints, one schema. Prompt engineering is cheaper than a training run.
  3. Add tools or retrieval for anything that must be true today.
  4. Measure. If you are already at the product bar, you are done.
  5. Fine-tune only the remainder: format, tone, label set, or a distilled small model.
type SftRow = {
  messages: Array<
    | { role: 'system'; content: string }
    | { role: 'user'; content: string }
    | { role: 'assistant'; content: string }
  >;
};

const keep: SftRow = {
  messages: [
    {
      role: 'system',
      content: 'Extract a support ticket as JSON. No prose.',
    },
    {
      role: 'user',
      content: 'Login 500 on /settings since 10:14 UTC. Account acme-19.',
    },
    {
      role: 'assistant',
      content: JSON.stringify({
        severity: 'high',
        area: 'auth',
        summary: 'Login 500 on /settings',
        accountId: 'acme-19',
      }),
    },
  ],
};

const doNotTrain: SftRow = {
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    {
      role: 'user',
      content: 'What is our refund window for annual plans?',
    },
    {
      // This will be wrong the day finance changes the policy.
      role: 'assistant',
      content: 'Annual plans can be refunded within 14 days.',
    },
  ],
};

The first row teaches a shape. The second row teaches a fact you should have retrieved.

Data that is not a dataset

A usable SFT set is boring:

  • Coverage. Every intent you will score in production, including “refuse” and “ask a clarifying question.”
  • One job per row. Mixing “write a haiku” with “emit ticket JSON” in the same run teaches neither.
  • No eval leakage. If the row is in the golden set, it is not in training.
  • No PII you would not store in logs. Fine-tunes get copied, snapshotted, and sometimes sent to a vendor.
  • Human-reviewed labels. Model-generated synthetic data is fine as a draft. Shipping it unreviewed is how you lock in hallucinations.

How many rows? Enough that a held-out slice is statistically less embarrassing than a vibe check. For a narrow JSON extractor, a few hundred clean examples can move the needle. For “sound like our brand on any topic,” you will keep being disappointed. Prefer a style guide in the prompt plus retrieval over a personality transplant.

Dumping Slack threads fails in a specific way: the model learns your mistakes at the same rate as your wins, and it learns yesterday’s product name.

Measure twice, train once

Log three numbers on the same golden set, same judge, same pinned model snapshot:

Slice What it tells you
Base + production prompt The baseline you must beat.
Base + prompt + few-shot / tools / RAG Whether you needed weights at all.
Fine-tuned + shorter prompt The thing you actually bought.

If the middle slice already wins, cancel the job. If the fine-tune wins on format but loses on factuality, you trained the wrong axis: put the facts back in retrieval and keep the small model for the schema.

After you ship:

  • Pin the checkpoint id in config, not “whatever is latest.”
  • Retrain when the label set changes, not when someone dislikes a comma.
  • Keep the previous checkpoint. Fine-tunes regress too.

Do not quote training prices. Count relative cost: a retrain cycle plus eval plus the ops of a custom model, versus tokens you would have spent on few-shot and retrieval. For many CRUD extractors, structured outputs plus a schema kill the need to train.

When you should still not fine-tune

  • The failure is retrieval. Wrong chunk in, confident wrong answer out. Train the index, not the weights.
  • The failure is math or freshness. Call SQL, not a memorized table.
  • The failure is permissions. No checkpoint will learn GDPR. Filter in code (permission-aware retrieval).
  • You cannot eval. Then you cannot tell a good run from a lucky one.
  • The provider does not offer it (or is sunsetting the UI). Prompt, tools, and RAG still exist.

Fine-tuning is the last lever on purpose. It creates a snowflake model you have to babysit. That can be worth it for a high-volume, stable format. It is not worth it because a blog said “custom GPTs convert better.”

Interview answer (out loud)

“I fine-tune when evals show the model already has the knowledge but will not hold a format, tone, or label set, and when I have hundreds of clean, de-duplicated examples that are not the eval set. I do not fine-tune to teach facts I can retrieve or compute. I run base, prompt-plus-tools, and the candidate checkpoint on the same golden set. If the prompt already clears the bar, I skip training and avoid owning another model version.”

Further reading (primary sources)

Related guides