Frontend Beauty
ESC

Type to search the knowledge base.

Tool Calling in Next.js

Run Next.js tool calling in a Route Handler: stream the UI, execute tools on the server, and keep secrets off the client.

intermediate6 min read
  • nextjs
  • tools
  • ai-sdk

A tool that hits Stripe from a 'use client' module is an incident write-up. The model did not charge the card. Your browser did, with whatever cookies and keys it could see. Next.js tool calling means the model emits a structured call, your Route Handler runs the function, and the client only renders “looked up order” or “refund proposed.”

The JSON is a proposal. Authorization, idempotency, and secrets live in the same process that already owns the database session.

Key takeaways

  • Tools execute on the server. The browser streams UI events, not provider credentials.
  • Validate args with Zod (or JSON Schema) before execute. The model will invent fields.
  • Auth the request first. Then pass a trusted userId into the tool. Do not let the model supply the actor.
  • Stream tool-invocation parts so the UI can show “calling getOrder” without waiting for the final sentence.
  • Cap steps. A tool loop without stopWhen is an unbounded bill.

Why the browser is the wrong process

Client components can call fetch('/api/chat'). They cannot hold STRIPE_SECRET_KEY, OPENAI_API_KEY, or a service-role database URL. Those values in a NEXT_PUBLIC_ env var are in the bundle. See secrets in frontend bundles and client vs server LLM calls.

Even a “harmless” searchDocs tool on the client is a CORS and quota problem: the embedding key leaks, and every tab shares one provider limit. Your origin is the place you can attach a user, a rate limit, and a trace id.

This is function calling applied to the App Router. The schema and the loop are the same. The process boundary is the part people skip.

Next.js tool calling belongs in a Route Handler

A Route Handler is an HTTP entrypoint. For chat, that is POST that reads UI messages, converts them to model messages, and streams a response. Tools hang off streamText. They run in this process, with process.env that never shipped to the browser.

// app/api/chat/route.ts
import { streamText, tool, convertToModelMessages, stepCountIs } from 'ai';
import { z } from 'zod';
import { getSession } from '@/lib/auth';
import { db } from '@/lib/db';

export const maxDuration = 30;

export async function POST(req: Request) {
  const session = await getSession(req);
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { messages } = await req.json();

  const result = streamText({
    model: 'openai/gpt-4.1-mini',
    system:
      'You help the signed-in user inspect their own orders. You cannot refund.',
    messages: convertToModelMessages(messages),
    stopWhen: stepCountIs(5),
    tools: {
      getOrder: tool({
        description: 'Load one order the current user owns',
        inputSchema: z.object({
          orderId: z.string().min(1).describe('Opaque order id, not email'),
        }),
        execute: async ({ orderId }) => {
          const order = await db.orders.findOwned(session.userId, orderId);
          if (!order) return { error: 'not_found' };
          return {
            id: order.id,
            status: order.status,
            totalCents: order.totalCents,
          };
        },
      }),
    },
  });

  return result.toUIMessageStreamResponse();
}

Notes that survive SDK version churn:

  • The helper names move (parameters vs inputSchema, maxSteps vs stopWhen, toDataStreamResponse vs toUIMessageStreamResponse). The shape does not: schema, execute, stream back to the UI.
  • execute closes over session.userId. The model cannot pass a different user.
  • Returning { error: 'not_found' } is better than throwing for “order does not exist.” Throws look like outages. Missing rows are data.

Auth, then side effects

Read tools still need a tenant. Write tools need a human or a second check. If you add refundOrder, the handler must:

  1. Confirm the session.
  2. Load the order as owned.
  3. Bound the amount.
  4. Use an idempotency key you generate, not one the model invented (or you accept the model key only after you bind it to userId + orderId server-side).
  5. Pause for approval if money moves. That is human in the loop, not a system-prompt plea.
async function refundOwnedOrder(
  userId: string,
  args: { orderId: string; amountCents: number },
) {
  const order = await db.orders.findOwned(userId, args.orderId);
  if (!order) throw new Error('not_found');
  if (args.amountCents > order.refundableCents) throw new Error('amount');
  // Failure: trusting args.userId from the model. There is no such field on purpose.
  return payments.refund({
    orderId: order.id,
    amountCents: args.amountCents,
    idempotencyKey: `refund:${userId}:${order.id}:${args.amountCents}`,
  });
}

If the tool must not run in this request (approval, queue, slack), omit execute and let the UI post a decision to a second route. The model sees a tool result only after your code says so.

What the client is allowed to render

Stream tool parts so the chat UI can show status without waiting for the final assistant sentence.

'use client';

import { useChat } from '@ai-sdk/react';

export function OrderChat() {
  const { messages, sendMessage, status, stop } = useChat();

  return (
    <ul>
      {messages.map((m) => (
        <li key={m.id}>
          {m.parts.map((part, i) => {
            if (part.type === 'text') return <span key={i}>{part.text}</span>;
            if (part.type === 'tool-getOrder') {
              return (
                <p key={i} data-state={part.state}>
                  {part.state === 'output-available'
                    ? `Order ${part.output?.id ?? ''}`
                    : 'Looking up order…'}
                </p>
              );
            }
            return null;
          })}
        </li>
      ))}
      {status === 'streaming' ? (
        <button type="button" onClick={() => stop()}>
          Stop
        </button>
      ) : null}
    </ul>
  );
}

Part type names vary by SDK version (tool-invocation vs tool-${name}). Render from part.type and a documented state field. Do not parse the assistant’s English for “I called getOrder.”

Stop still needs AbortController semantics: aborting a stream should not leave a refund half-applied. If execute already ran, abort cancels generation, not the side effect. Make writes idempotent and short.

Caps, parallelism, and retries

Knob What breaks if you skip it
stopWhen / max steps The model retries a failing tool until the bill notices
Timeouts on execute A hung HTTP call holds the stream open
Allowlist of tool names A future prompt adds runSql and someone ships it
No retry on 2xx writes “Try again” double-refunds

Parallel tool calls are normal: the model may emit getOrder twice. Your execute must tolerate that. Do not open two refunds because two calls arrived in one step.

Keep the tool list small. A kitchen-sink run with a string argument is how you invent incident review. Schema design is a sibling page. Here the Next.js-specific rule is: if it is not imported in the Route Handler, it cannot run.

Footguns

  • Passing messages from the client straight into a privileged tool. The client can spoof tool results. Re-validate or ignore client-supplied tool rows unless you signed them.
  • Running tools in Edge with Node-only SDKs. Stripe and pg often need Node. Set the runtime explicitly or you will debug crypto in the wrong place.
  • Logging full tool args. Order ids are fine. Card PANs and emails are not. Redact in traces.
  • Letting the model pick the database. One getOrder. Not rawQuery.

Interview answer (out loud)

“In Next.js I put tool calling in a Route Handler. The client streams from my origin. The model returns a typed call, Zod validates it, and execute runs with the session user id I already authenticated. Secrets never enter the bundle. I cap steps, I do not retry successful writes, and the UI renders tool parts as status, not as trusted English. A refund is still my code, with an idempotency key, after policy.”

Further reading (primary sources)

Related guides