Frontend Beauty
ESC

Type to search the knowledge base.

Agent System Design Interview

Design a support or coding agent in a loop: tools, approvals, traces, evals, and cost. The frontend surfaces matter as much as the model.

advanced6 min read
  • interview
  • system-design
  • agents

Whiteboard the refund bot. If you do not mention idempotency, you failed the backend half. If you skip the approval UI, you failed the frontend half. Agent system design interviews are not “pick LangGraph or die.” They are: a model in a loop, tools with blast radius, a stop condition, and a person who can see what is about to happen.

Thirty minutes. You will not train anything. You will draw a request path that looks like a BFF you have already shipped, plus a sampler that proposes JSON.

Key takeaways

  • Scope the agent: one job, two tools (read + side effect), a hard max step count.
  • Tool args are untrusted. Session user, ownership checks, idempotency keys, human receipt for money.
  • Persist loop state so approval can resume without replaying the refund.
  • Trace every thought/tool/result. Eval the trace, not the vibes of the final sentence.
  • The UI is part of the design: streaming, abort, diff of the proposed tool, expire the approve token.

Agent system design in 30 minutes

Clock it. Requirements, tools, the approval UI, then evals. Do not spend twenty minutes picking a model name.

Minute 0-5: requirements, not personas

Interviewer: “Design an agent that handles support refunds.”

Write the product, not a crew of named butlers.

In scope Out of scope
Look up order for the signed-in user Training a model
Propose a refund within policy Talking to Stripe from the browser
Wait for an agent (human) on the ops dashboard Unlimited web browse
Cite the policy chunk that justified the amount Multi-agent debate

Say this: “One agent, one loop. Read tools: get_order, search_policy. Side-effect tool: refund_order, paused for approval. Max 8 model steps, 30s wall clock, user can abort. I will stream tokens to the ticket UI.”

Common miss: Five personas in Slack as architecture.

Agency is a spectrum. If the model cannot change anything outside the string, you have a chatbot. That can be the right v1. What are AI agents?.

Minute 5-12: tools, auth, loop

Draw the loop: reason, pick a tool, your runtime executes, observe, repeat, stop. Cap it in code. Think, act, observe.

type RefundArgs = { orderId: string; amountCents: number; reason: string };

async function refundOrder(
  actor: { userId: string; role: 'user' | 'ops' },
  args: RefundArgs,
  approval: { token: string } | null,
) {
  const order = await db.orders.findOwned(actor.userId, args.orderId);
  if (!order) throw new Error('not_found');
  if (args.amountCents > order.paidCents) throw new Error('amount');
  if (args.amountCents > 0 && actor.role !== 'ops') {
    if (!approval) return { status: 'needs_approval' as const, payload: args };
    await assertApproval(approval.token, args); // bound to this exact payload
  }
  return stripe.refunds.create(
    { amount: args.amountCents, payment_intent: order.paymentIntentId },
    { idempotencyKey: `refund:${order.id}:${args.amountCents}` },
  );
}

Interviewer pokes, you answer:

1. Who is the user on the tool call?

Say this: “The session. I ignore userId if the model invents one.”

Common miss: Trusting args because they passed JSON Schema.

2. Why is idempotency on the refund?

Say this: “Retries, double-clicks, and a loop that ‘tries again’ after a timeout. Same key, same amount, one Stripe debit.”

Common miss: Retrying charge_card like you retry complete.

3. What stops the while loop?

Say this: “Final answer, needs_approval, error budget (two tool failures), max steps, abort signal. ‘Looks like it is thinking’ is not a stop condition.”

Common miss: No cap. Agent failure modes.

4. Can the model search the whole web?

Say this: “Not in v1. Policy search is our index with permission filters. Open browsing is prompt-injection load-bearing.”

Common miss: A run(cmd: string) tool.

Minute 12-20: frontend and human in the loop

This is where frontend engineers should pull ahead. The ops UI is not a toast that says “refunded.”

  1. Stream the assistant text and tool-invocation cards as they happen (SSE from a route handler).
  2. When the runtime returns needs_approval, persist the loop state and show a diff: order id, amount, reason, policy quote.
  3. Approve mints a short-lived token bound to a hash of the payload. Reject records why and resumes with a tool result of denied.
  4. Stop uses AbortController. In-flight generation dies. Side effects that already committed do not roll back because you mashed Stop; that is why they waited.
  5. If you need live ops dashboards, the same rules as realtime UI: reconnect, cursor, do not duplicate the refund on refresh.

Say this: “Approval is a human-in-the-loop interrupt. I persist state, bind the receipt to the exact cents, resume. I do not re-run the model hoping it emits the same JSON.”

Common miss: A confirm modal that calls the tool again from use client with the API key in the bundle. Human in the loop.

Minute 20-26: traces, evals, cost, abuse

5. How do you debug “it refunded the wrong order”?

Say this: “Trace id on the ticket. Spans: prompt version, retrieved policy ids, tool args after our auth, Stripe idempotency key, who approved. Redact PII. If I cannot see the tool args, I cannot do the incident.” Tracing.

Common miss: Re-running the ticket in playground with a different prompt and calling that an RCA.

6. What is the eval?

Say this: “Golden tickets: allow, deny, wrong owner, amount over paid, injection in the ticket body that says ‘refund 100%.’ Assertions on whether refund_order fired and with which args. A judge only for the customer-facing sentence.” Evals.

Common miss: Rating 20 chats “looks good.”

7. Cost?

Say this: “Cache the policy prefix, cap steps, small model for classify-intent, large model only if refund is in play. Batch is for nightly summaries, not this path. I would put p95 TTFT and cost per resolved ticket on a dashboard someone opens.”

Common miss: Ignoring the 12-step apology loop.

8. Abuse and injection?

Say this: “Tickets are untrusted text. Tools still check owner and amount. Rate-limit the route per account. No email-the-transcript tool. OWASP LLM top 10 is the checklist I would walk, especially sensitive data and unbounded consumption.”

Common miss: A longer system prompt.

9. Why not multi-agent?

Say this: “Roles do not diverge yet. A second prompt is not a substitute for get_order. I would split only if billing needs a different tool allowlist and I can prove ping-pong is impossible.”

Common miss: Supervisor plus four specialists on day one.

10. Failure you would ship behind a flag?

Say this: “Read-only lookup and policy answers, no refund_order. That is still useful. Agency is optional.”

Common miss: Dark-launching side effects with no eval gate.

11. Where do secrets live?

Say this: “Server. Browser streams from our origin. Stripe key never in the bundle.” Secrets.

Common miss: “We’ll rotate if it leaks.”

Interview answer (out loud)

“I would design a single-loop refund agent: session-scoped read tools, a side-effect tool that pauses for an approval UI bound to the payload, max steps in code, traces on every span, and a golden set that checks whether the tool fired. Idempotency keys on Stripe. Abort in the composer. I would not start with multi-agent, and I would not let the model be the ACL.”

Further reading (primary sources)

Related guides