Frontend Beauty
ESC

Type to search the knowledge base.

Function Calling and Tools

Tool calling is the model emitting a structured call you execute. You run the function. The model never gets a shell unless you hand it one.

intermediate5 min read
  • tools
  • function-calling
  • agents

The JSON said refund_order. Your code still has to check the user owns that order. Function calling (tool calling) is the model emitting a structured proposal. It is not the model reaching through the API into Stripe. If you hand it a shell, that is a product choice, and a bad default.

Treat the call like a form POST from an untrusted client: validate the shape, authorize the actor, run your function, return a small result. Then, if the loop continues, the model sees the observation and samples again.

Key takeaways

  • The model emits a name plus JSON args. Your process runs the function.
  • Validate args against the schema before any side effect.
  • Authorization lives in the handler. The JSON is a proposal.
  • Parallel tool calls need per-call success and failure observations.
  • The model never gets a shell, a DB connection, or a secret unless you pass one.

Function calling is a proposal, not a permission

A typical round trip, names aside (Chat Completions tools, Anthropic tool_use blocks, AI SDK tools):

  1. You send tool schemas with the messages.
  2. The model returns one or more calls: name, arguments, a call_id.
  3. You parse arguments. If parse fails, you return a tool error observation, not a crash.
  4. You run the handler with the request’s user, not a user id the model invented.
  5. You send tool results back. The model may call again or answer.
type ToolName = 'get_order' | 'refund_order';

type ToolCall = {
  id: string;
  name: ToolName;
  args: unknown;
};

type AuthedUser = { id: string; role: 'user' | 'admin' };

function asObject(args: unknown): Record<string, unknown> {
  if (typeof args !== 'object' || args === null) throw new Error('args');
  return args as Record<string, unknown>;
}

function parseOrderId(args: unknown): { orderId: string } {
  const { orderId } = asObject(args);
  if (typeof orderId !== 'string' || !/^ord_[a-z0-9]+$/.test(orderId)) {
    throw new Error('orderId');
  }
  return { orderId };
}

function parseRefundArgs(args: unknown): { orderId: string; amountCents: number } {
  const { orderId } = parseOrderId(args);
  const { amountCents } = asObject(args);
  if (typeof amountCents !== 'number' || amountCents <= 0 || amountCents > 50_000) {
    throw new Error('amountCents');
  }
  return { orderId, amountCents };
}

Use Zod if you already have it; the point is the parse happens before stripe. Schematic types drift across providers. The invariant does not: untrusted JSON in, typed object or a tool error out.

Schema in, your runtime, result back

Handlers close over the authenticated user from the route, the same way a Next.js route handler already does for non-LLM traffic.

async function executeTool(user: AuthedUser, call: ToolCall): Promise<string> {
  switch (call.name) {
    case 'get_order': {
      const { orderId } = parseOrderId(call.args);
      const order = await db.orders.findOwned(user.id, orderId);
      if (!order) return JSON.stringify({ error: 'not_found' });
      return JSON.stringify({
        id: order.id,
        status: order.status,
        paidCents: order.paidCents,
      });
    }
    case 'refund_order': {
      const args = parseRefundArgs(call.args);
      const order = await db.orders.findOwned(user.id, args.orderId);
      if (!order) return JSON.stringify({ error: 'not_found' });
      if (args.amountCents > order.paidCents) {
        return JSON.stringify({ error: 'amount' });
      }
      // Human approval belongs here for irreversible money movement.
      const refund = await refundProvider.credit(order.id, args.amountCents);
      return JSON.stringify({ refundId: refund.id, status: refund.status });
    }
    default: {
      const _exhaustive: never = call.name;
      return JSON.stringify({ error: 'unknown_tool', name: _exhaustive });
    }
  }
}

Notes that bite in production:

  • Return small JSON. A 40k HTML dump as a tool result will dominate the next turn and hide the field you needed.
  • Do not echo secrets. If the provider response included an API key, strip it. Tool results are prompt text on the next call.
  • Errors are data. { error: 'not_found' } is an observation. Throwing out of the loop looks like your app crashed. The model cannot recover from a 500 it never sees.
  • Unknown tool names. The model will invent refund_all. The default branch must not map that to a wildcard executor.

The Next.js-shaped version of this (stream UI, tools on the server) is tool calling in Next.js. The schema craft is tool schema design.

Parallel calls and partial failure

One completion can request get_order and get_account_credit together. Run independent reads concurrently. Do not abort the batch because one id 404’d.

async function executeParallel(
  user: AuthedUser,
  calls: ToolCall[],
): Promise<Array<{ callId: string; content: string }>> {
  const settled = await Promise.allSettled(
    calls.map((call) => executeTool(user, call)),
  );
  return settled.map((result, i) => ({
    callId: calls[i].id,
    content:
      result.status === 'fulfilled'
        ? result.value
        : JSON.stringify({ error: 'tool_failed' }),
  }));
}

Writes are different. Two refund_order calls in one turn is a smell. Either reject the second, or require an idempotency key and serialize on orderId. Retrying a completion after a 503 must not double-credit: the write path needs a key the model cannot “creatively” change, or you mint the key server-side and ignore the model’s.

Auth lives in the handler

The model will put userId in the args if you give it that field. Do not give it that field. Take the user from the session cookie / JWT on your origin. The browser talks to your server; keys stay off the bundle (secrets in frontend bundles, client vs server LLM calls).

Same rule as a REST handler:

  • findOwned(user.id, orderId), never find(orderId) plus a prompt that says “only refund your own.”
  • Admin tools are a separate schema, gated by user.role, not by “you are an admin assistant.”
  • Prompt injection in a retrieved page can ask the model to call refund_order. If the handler checks ownership, the blast radius is a failed call. If the handler trusts args, the blast radius is money.

You still need the think, act, observe loop to cap how many proposals you will even look at.

Interview answer (out loud)

“Function calling is the model emitting a structured tool call that my code may run. I validate arguments, authorize with the session user, execute in my process, and return a small JSON observation. Parallel reads can settle independently. Writes need ownership checks and idempotency. The model never gets a shell unless I explicitly sandbox one. The JSON is a proposal, not a permission.”

Further reading (primary sources)

Related guides