Frontend Beauty
ESC

Type to search the knowledge base.

Vercel AI SDK Fundamentals

The AI SDK gives TypeScript apps streaming, tools, and UI hooks without hand-rolling SSE. Where it helps, and where you still own the server.

beginner5 min read
  • ai-sdk
  • typescript
  • nextjs

You can wire useChat in an afternoon and have tokens painting in the browser. That demo is not a product. The Vercel AI SDK removes the SSE boilerplate. You still need auth on the route, a key that is not in the bundle, traces, and a plan for abort. The SDK is a TypeScript client for models. It is not your backend.

Hook names and stream helpers move between major versions. The architecture does not: a server function talks to the provider, a UI hook consumes the stream, tools run where secrets already live.

Key takeaways

  • AI SDK Core (streamText, generateText, generateObject) is the provider-agnostic server API.
  • AI SDK UI (useChat and friends) is React state plus a stream protocol. It does not hold the API key.
  • Route Handlers (or other server endpoints) are the BFF. Auth, redaction, and tool execution live there.
  • Provider packages are adapters. Swap models without rewriting the chat list.
  • The SDK will not invent evals, ACLs, or cost caps. Those stay yours.

What the Vercel AI SDK actually owns

Three layers:

Layer You import Job
Core ai Turn messages into a generation, tools, structured objects
UI @ai-sdk/react Chat state, streaming parts, stop
Providers @ai-sdk/openai, etc. Model ids and auth headers on the server

There is also a transport object in recent UI versions (DefaultChatTransport) so the hook does not hard-code /api/chat forever. Treat that as plumbing.

What you still write:

  • Session checks (the model does not know your user).
  • Prompt versions and retrieval.
  • Tool resolvers that hit your DB with your authorization.
  • Logging. experimental_telemetry exists in some versions; a trace id is still your invariant.

If the key is in NEXT_PUBLIC_* or any client module, stop and read secrets in frontend bundles. The browser streams from your origin. See client vs server LLM calls.

A route that streams

App Router Route Handler, schematic against current Core APIs. Pin versions in the app; do not copy field names into a spec.

// app/api/chat/route.ts
import { convertToModelMessages, streamText, type UIMessage } from 'ai';
import { auth } from '@/lib/auth';

export const maxDuration = 30;

export async function POST(req: Request) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: 'provider/chat-small',
    system:
      'Answer from the supplied docs. If they are missing, say you do not know.',
    messages: await convertToModelMessages(messages),
    abortSignal: req.signal,
    maxRetries: 2,
  });

  return result.toUIMessageStreamResponse();
}

Notes that bite people:

  • maxDuration (or your host’s equivalent) must exceed the stream, or the platform kills the response mid-sentence.
  • abortSignal: req.signal is how Stop from the browser cancels the provider call. Without it, useChat’s stop is cosmetic.
  • convertToModelMessages exists because UI messages (parts, tool chips) are not the provider messages array. Do not JSON.stringify the React state into Chat Completions and hope.
  • Auth first. An open /api/chat is a public wallet.

Tools belong on this route, not in a 'use client' module. Tool calling in Next.js is the dedicated walkthrough. Same for structured output via generateObject.

useChat without putting keys in the bundle

'use client';

import { useChat } from '@ai-sdk/react';
import { DefaultChatTransport } from 'ai';
import { useState } from 'react';

export function Chat() {
  const [input, setInput] = useState('');
  const { messages, sendMessage, stop, status } = useChat({
    transport: new DefaultChatTransport({ api: '/api/chat' }),
  });

  const busy = status === 'submitted' || status === 'streaming';

  return (
    <div>
      <ul>
        {messages.map((m) => (
          <li key={m.id}>
            <strong>{m.role}</strong>
            {m.parts.map((part, i) =>
              part.type === 'text' ? <span key={i}>{part.text}</span> : null,
            )}
          </li>
        ))}
      </ul>
      <form
        onSubmit={(e) => {
          e.preventDefault();
          if (!input.trim() || busy) return;
          sendMessage({ text: input });
          setInput('');
        }}
      >
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          disabled={busy}
        />
        <button type="submit" disabled={busy}>
          Send
        </button>
        <button type="button" onClick={stop} disabled={!busy}>
          Stop
        </button>
      </form>
    </div>
  );
}

Older examples used handleSubmit and stored input inside the hook. Newer ones do not. Read the version you installed. The product rules stay: disable Send while streaming, keep Stop, render parts so tool calls are not invisible.

For a less janky list, streaming chat UI in React. For the HTTP shape behind the hook, streaming LLM responses.

What the SDK will not do for you

  • Authorization of tools. A Zod schema on refundOrder is not “does this user own the order.”
  • Retrieval ACLs. If you pass docs into system, you chose them. The SDK will stream the leak as cheerfully as any other token.
  • Rate limits. useChat will fire every time the user submits. Gate the route (rate limits from the browser).
  • Evals. Shipping a prompt from a local demo still needs a golden set.
  • Stable UX on a bad day. Set retries and timeouts on the server. Surface 429s instead of spinning.

Provider adapters are the actual win versus hand-rolled fetch to one vendor: you keep streamText and change model. Schema compatibility is not guaranteed. A tool definition that one provider accepts may need a tighter JSON Schema for another. Test the route, not only the playground.

generateText is the non-streaming cousin: cron jobs, extractors, eval runners. Use it when the UI does not need tokens. Use streamText when a person is staring at a bubble.

Interview answer (out loud)

“The Vercel AI SDK is a TypeScript layer: Core for streamText and tools on the server, UI hooks like useChat for the chat list. The browser talks to my Route Handler, not to the model provider, so the API key never ships in the bundle. I authenticate that route, pass the abort signal, and run tools with the same authorization I would use without an SDK. I still own evals, rate limits, and retrieval filters. The SDK saves me from hand-rolling SSE, not from product engineering.”

Further reading (primary sources)

Related guides