Frontend Beauty
ESC

Type to search the knowledge base.

Client vs Server LLM Calls

An LLM API key in the browser is public. Stream from your origin, quota per user, and treat client SDKs as an explicit product choice.

beginner6 min read
  • security
  • architecture
  • frontend

Anything in the JavaScript bundle is public. That includes sk- and the Anthropic key you “hid” in a Vite env without the VITE_ prefix… until a teammate added the prefix to “make it work.” An LLM API key in the browser is not a configuration mistake. It is a published credential with your quota attached.

The browser talks to your origin. Your server talks to the provider. The only exception is a product that intends to spend the user’s own key, with UI that says so.

Key takeaways

  • Provider secrets stay on the server. NEXT_PUBLIC_, VITE_, and REACT_APP_ are compile-time inserts into client JS.
  • The BFF pattern: browser streams from your Route Handler, you call the model, you attach a user id and a budget.
  • CORS will not save you. If a key works in DevTools, it works from a random origin.
  • Bring-your-own-key is a product mode, not a workaround for missing env hygiene.
  • Short-lived client tokens (if a provider issues them) still need a server that mints and revokes them.

An LLM API key in the browser is a published secret

Frameworks document this. They still get ignored because “it ran locally.”

// Client bundle. Anyone can read this in Sources.
const leaked = process.env.NEXT_PUBLIC_OPENAI_API_KEY;

// Server module. Fine, as long as this file is never imported from a client component.
const ok = process.env.OPENAI_API_KEY;

Next.js environment variables inlined NEXT_PUBLIC_* at build time. Vite does the same for VITE_*. There is no “obscure the name” step. See secrets in frontend bundles for the general rule. This page is that rule applied to model providers: keys are also quotas, so a leak is both a security event and a billing event.

A client component that imports a module which imports openai at the top level can pull the SDK (and sometimes the key) into the client graph. Treat “server-only” as a lint rule, not a hope.

import 'server-only';
import { generateText } from 'ai';

export async function completeOnServer(prompt: string) {
  return generateText({
    model: 'openai/gpt-4.1-mini',
    prompt,
  });
}

If a 'use client' file imports completeOnServer, the build should fail. If it does not fail, you do not have a boundary.

BFF: your origin, your quota

The default architecture for a React app:

// app/api/chat/route.ts
import { streamText, convertToModelMessages } from 'ai';
import { getSession } from '@/lib/auth';
import { assertUserBudget } from '@/lib/quota';

export async function POST(req: Request) {
  const session = await getSession(req);
  if (!session) return new Response('Unauthorized', { status: 401 });
  await assertUserBudget(session.userId);

  const { messages } = await req.json();
  const result = streamText({
    model: 'openai/gpt-4.1-mini',
    messages: convertToModelMessages(messages),
  });
  return result.toUIMessageStreamResponse();
}

The browser’s fetch('/api/chat') is same-origin. Cookies work. You can rate-limit per user. You can log a trace id. You can abort when the tab closes. That is a Route Handler, not a “proxy” you are ashamed of.

Call site Key location Quota you can enforce Use
Browser → provider Bundle or typed-in key Provider’s, shared Almost never
Browser → your origin → provider Server env Per user, per session Default
Browser → provider with user-owned key User’s key, never stored (or stored encrypted at their request) User’s provider quota BYOK product
Browser → provider with minted ephemeral token Server mints, client holds a TTL’d token Whatever the token encodes Realtime / WebRTC-style APIs

Tools that touch Stripe, email, or the database cannot run in the browser even if the model key is ephemeral. That is tool calling in Next.js.

When a client SDK is actually OK

A few honest cases:

  1. BYOK. The settings page asks for the user’s key. You send it to the provider from the browser or you send it to your server and you treat it as a secret you did not want. Prefer the server even then: you can redact logs, attach a proxy allowlist, and avoid CORS fights. If you truly never want the key on your disk, say that in the UI, do not persist it, and expect DevTools to show it. That is the product.
  2. Publishable / constrained tokens. Some providers mint a short-lived client secret scoped to a session. Your server creates it after auth. The browser uses it for a realtime channel. The minting endpoint is still yours. Rotation and revocation are still yours.
  3. Local models. http://127.0.0.1:11434 from a desktop app is a user-owned runtime. A public website calling a visitor’s localhost is a different threat model (and usually blocked).

“The SDK has a browser build” is not a case. That means the HTTP client runs in JS, not that the secret belongs there.

CORS is not access control

If the provider allowed browser origins, an attacker would call it with your leaked key from their page. If the provider does not allow CORS, people “fix” it with a browser extension or a local proxy, then ship the key anyway. Either way, CORS is a browser rule for reads from JS. It is not a permission system. Server-side calls do not use CORS.

Same-origin to /api/chat is the setup you want. Then your handler decides who may spend tokens.

Quota belongs next to the key

A leaked org key has one bucket: whoever holds it. A BFF has many:

  • Per authenticated user
  • Per anonymous session cookie
  • Per IP as a backstop, knowing NATs exist
  • Daily cap vs burst cap

The provider’s 429 is a last line, not a product. Rate limits from the browser covers the UX. The architecture point is: you cannot attach a user to a request the browser sent straight to OpenAI with a shared key.

OpenAI production best practices put keys in server environments and talk about tracking usage. That is this split.

Footguns

  • Preview deployments with real keys. A Vercel preview URL is still a website. Restrict who can hit /api/chat, or use a separate provider project with a tiny cap.
  • Logging prompts plus keys. Redact Authorization. Assume the prompt has PII.
  • Importing the provider SDK in a shared lib/ai.ts used by both client and server. Split the package.
  • Putting the key in localStorage “just for the demo.” Demos get screenshotted. Demos get cloned.

Interview answer (out loud)

“The LLM API key never goes in the browser bundle. NEXT_PUBLIC and VITE prefixes are public. The client streams from my origin. The Route Handler authenticates, checks a per-user budget, and calls the provider. CORS is not a secret. Bring-your-own-key is an explicit mode where the user knows they are spending their quota. Tools with side effects still run on the server.”

Further reading (primary sources)

Related guides