Frontend Beauty
ESC

Type to search the knowledge base.

LLMs for Frontend Engineers

Treat a large language model like a weird HTTP API: tokens in, tokens out, no memory, no guarantees. What that means for UI, retries, and types.

beginner6 min read
  • llms
  • api
  • frontend

You already know fetch. A large language model call is the same shape: a URL, headers, a JSON body, a status code, bytes back. The difference sits in the response. The body is a sample from a probability distribution, not a row from Postgres.

Same request on Tuesday can differ from Monday. There is no session on the model. There is no “the user is logged in.” There is a list of tokens you send, a list of tokens you get, and usage metadata. Everything you care about (auth, memory, idempotency, stop) lives in your code.

That mapping is the whole job for a frontend engineer. You do not need transformer math to ship a chat box. You do need to stop treating the API like a typed RPC.

Key takeaways

  • A large language model is a next-token sampler behind HTTP. It does not remember, authorize, or tell the truth.
  • The browser talks to your origin. Keys stay on the server. The client owns abort, retries, and loading.
  • Types describe the request. They do not make the completion stable.
  • Retrying a 429 is reasonable. Retrying a successful tool call is how you double-apply a side effect.

Same request, different Tuesday

A completion is not SELECT. Given a prompt, the model assigns a probability to every next token, draws one, then repeats until it hits a stop. Temperature and other sampler knobs change that draw. Even with those knobs pinned, providers do not promise bit-identical output across regions, batches, or days.

So the TypeScript type on the response is a shape, not a contract about content:

type ChatMessage = {
  role: 'system' | 'user' | 'assistant';
  content: string;
};

type CompletionOk = {
  message: ChatMessage;
  usage: { inputTokens: number; outputTokens: number };
  finishReason: 'stop' | 'length' | 'content_filter' | string;
};

// Failure mode: you typed `invoiceId: string` and the model wrote "N/A".

finishReason: 'length' means you hit max_tokens with a half-sentence. Treat that as a truncated payload, not a full answer. Do not parse it as if the object closed cleanly.

Map a large language model onto fetch

The useful mental model is a BFF route, not a client SDK in the bundle.

export async function POST(req: Request) {
  const apiKey = process.env.LLM_API_KEY;
  if (!apiKey) {
    return Response.json({ error: 'server_misconfigured' }, { status: 500 });
  }

  const body = (await req.json()) as {
    messages: ChatMessage[];
    maxTokens?: number;
  };

  const upstream = await fetch('https://api.example.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      authorization: `Bearer ${apiKey}`,
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      model: process.env.LLM_MODEL,
      messages: body.messages,
      max_tokens: body.maxTokens ?? 512,
    }),
    signal: req.signal,
  });

  if (!upstream.ok) {
    return Response.json(
      { error: 'upstream', status: upstream.status },
      { status: 502 },
    );
  }

  const data = (await upstream.json()) as CompletionOk;
  return Response.json(data);
}

Forward req.signal so a client abort actually cancels the upstream work. That is the same AbortController pattern you already use for search.

Two rules that match every other third-party API:

  1. No secrets in the bundle. VITE_ / NEXT_PUBLIC_ prefixes leak. See secrets in frontend bundles and client vs server LLM calls.
  2. CORS is not your friend here. The browser should not talk to the model vendor. Your origin should. See CORS if you are tempted to “just add the header.”

Types that survive a sample

Type the messages you send. Validate the object you get if you need fields. A string completion is untrusted text: it can contain markdown, HTML, or instructions aimed at the next turn. Rendering it as HTML is an XSS bug with extra fluency.

function assertAssistantText(value: unknown): string {
  if (
    typeof value !== 'object' ||
    value === null ||
    !('message' in value) ||
    typeof (value as CompletionOk).message.content !== 'string'
  ) {
    throw new Error('unexpected completion shape');
  }
  return (value as CompletionOk).message.content;
}

If you need an invoice object, do not regex the prose. Use structured outputs. If you need the model to do something, that is a tool proposal your server executes after auth, not a string you eval.

Retries are not free, and they are not idempotent

HTTP retries assume the same request yields the same resource. LLM generation does not. A retry is a new sample. That is fine for a summarizer. It is not fine if the first attempt already called create_ticket.

Situation Retry?
429 / 503 before any bytes Yes, with jitter, bounded attempts
Timeout with no body Maybe. You do not know if a tool ran
200 with finishReason: 'stop' No. You already paid for a completion
Stream already painted tokens No. The user saw a partial answer

Cap max_tokens. Cap parallel requests per session. A retry loop that “keeps trying” is how a demo bill becomes a production incident.

The request also has no memory. If you omit prior turns, the model does not “still know.” You resend chat messages every time, and you pay for them. That is why tokens and context windows are a budget, not a trivia fact.

What the UI has to own

The model will not:

  • Disable Send while a request is in flight
  • Honor Stop
  • Deduplicate a double-click
  • Keep a conversation id stable across refreshes
  • Tell you the answer is grounded

You own those. Streaming vs waiting for the full JSON is a UX choice with a latency cost; the protocol is still HTTP. Timeouts belong on the server and the client. A hung fetch with no abort is a stuck composer.

Treat the assistant bubble as untrusted. Cite sources in your UI only if your code checked the ids. Do not let the model invent a URL and render it as a chip.

What still breaks

Hidden state. Putting “remember the user prefers dark mode” in the system prompt does not persist across devices. Store preferences in your database. Send them as data.

Optimistic UI on a sample. Showing “Ticket created” because the model said so, before your tool handler returned, is a lie. Wait for your status.

Schema drift across vendors. Chat Completions messages, Anthropic system plus messages, and newer “responses” APIs do not share field names. Keep an adapter on the server. Do not leak vendor types into React props.

Non-determinism in tests. Snapshotting a completion string will flake. Assert on structured fields, tool args, or a golden set, not on the poem.

Out loud, in an interview

“A large language model is an HTTP API that samples the next token. It has no memory and no permissions. I would put the key on the server, type the messages array, abort with the same AbortController I use for search, and never retry a side-effecting tool. Types describe the wire format. They do not make the answer true. If I need fields, I constrain the output. If I need an action, my code runs it after auth.”

Further reading (primary sources)

Related guides