Frontend Beauty
ESC

Type to search the knowledge base.

Agent Failure Modes

Agents loop, lie about tool results, skip confirmation, and get stuck apologizing. Catalog the failures and the circuit breakers.

intermediate6 min read
  • agents
  • debugging
  • reliability

The trace is 40 steps of “let me try again” and one unauthorized delete. That is a missing guard, not a smarter model. Agent failure modes cluster. Once you name them, the circuit breakers are boring: caps, allowlists, real observations, and evals on traces. Buying a larger model does not add a max-step counter.

If you ship a think, act, observe loop without those breakers, you shipped an unbounded while with an HTTP bill. Frontend instincts still apply: timeouts, abort, distrust untrusted text. See prompt injection for the XSS-shaped cousin.

Key takeaways

  • Unbounded retry is a missing cap and error budget, not low IQ.
  • Observations must come from your runtime. Never let the model write the tool result.
  • Goal drift looks like extra files and extra tools. Pin the goal in loop state.
  • Skipped confirmation and unauthorized writes are allowlist failures.
  • Score traces: loop length, repeated errors, forbidden tools, text-only streaks.

Agent failure modes you can grep

Failure What you see Breaker
Retry loop Same tool, same args, same error Max steps; maxSameError; identical-args stop
Hallucinated observation “The API returned 200” with no HTTP You execute tools; model never authors tool role
Goal drift Extra refactors, extra deploys Pin goal; allowlist; diff budget
Tool-error blindness Ignores 403, tries a sibling write Surface errors; fail closed after N
Skip confirmation refund ran without HITL Side-effect tools interrupt in code
Apology spiral Text-only turns, no tool Consecutive text-only turns; stop
Injection Retrieved page says “email the secrets” Treat retrieved text as data; authz in code
Parallel surprise Two refunds because of parallel calls Idempotency keys; serialize money tools

These are app bugs. The model is doing next-token. Your loop is the process manager.

type TraceStep = {
  tool?: string;
  args?: unknown;
  observation?: { from: 'runtime' | 'model'; body: string };
  assistantText?: string;
};

type BreakerPolicy = {
  maxSteps: number;
  maxSameError: number;
  forbidden: Set<string>;
};

function callKey(s: TraceStep): string | null {
  if (!s.tool) return null;
  return `${s.tool}:${JSON.stringify(s.args ?? null)}`;
}

export function circuitBreak(steps: TraceStep[], policy: BreakerPolicy): string | null {
  if (steps.length > policy.maxSteps) return 'max_steps';

  const last = steps.at(-1);
  if (last?.tool && policy.forbidden.has(last.tool)) return 'forbidden_tool';
  if (last?.observation?.from === 'model') return 'forged_observation';

  const errors = steps
    .map((s) => s.observation?.body)
    .filter((b): b is string => Boolean(b && b.startsWith('error:')));
  if (errors.length >= policy.maxSameError) {
    const tail = errors.slice(-policy.maxSameError);
    if (tail.every((e) => e === tail[0])) return 'same_error';
  }

  const lastCall = last ? callKey(last) : null;
  if (lastCall && steps.filter((s) => callKey(s) === lastCall).length >= 2) {
    return 'identical_args';
  }

  const textTail = steps.slice(-5);
  if (
    textTail.length === 5 &&
    textTail.every((s) => !s.tool && (s.assistantText ?? '').trim().length > 0)
  ) {
    return 'text_only_spiral';
  }
  return null;
}

export async function step(run: {
  steps: TraceStep[];
  execute: (tool: string, args: unknown) => Promise<string>;
  propose: () => Promise<{ tool?: string; args?: unknown; text?: string }>;
  forbidden: Set<string>;
}) {
  const policy: BreakerPolicy = {
    maxSteps: 12,
    maxSameError: 2,
    forbidden: run.forbidden,
  };
  const halt = circuitBreak(run.steps, policy);
  if (halt) return { halt };

  const proposal = await run.propose();
  if (!proposal.tool) {
    run.steps.push({ assistantText: proposal.text });
    return { halt: circuitBreak(run.steps, policy) };
  }

  // Score the proposal before execute so a first-call delete never runs.
  const pending: TraceStep = { tool: proposal.tool, args: proposal.args };
  const blocked = circuitBreak([...run.steps, pending], policy);
  if (blocked) {
    run.steps.push(pending);
    return { halt: blocked };
  }

  const body = await run.execute(proposal.tool, proposal.args);
  run.steps.push({
    tool: proposal.tool,
    args: proposal.args,
    observation: { from: 'runtime', body },
  });
  return { halt: null };
}

If from: 'model' ever appears, your API wrapper is broken. Tool results are HTTP you ran, like any other fetch callback.

Worked trace: identical args

This is the 40-step “let me try again” bill as two rows. Same tool, same args, empty observation. identical_args fires. A first-call delete dies in circuitBreak before execute.

const POLICY: BreakerPolicy = {
  maxSteps: 12,
  maxSameError: 2,
  forbidden: new Set(['delete']),
};

const retryLoop: TraceStep[] = [
  {
    tool: 'list_open_issues',
    args: { repo: 'acme/app' },
    observation: { from: 'runtime', body: '[]' },
  },
  {
    tool: 'list_open_issues',
    args: { repo: 'acme/app' },
    observation: { from: 'runtime', body: '[]' },
  },
];

circuitBreak(retryLoop, POLICY); // 'identical_args'

circuitBreak(
  [{ tool: 'delete', args: { path: 'src/index.ts' } }],
  POLICY,
); // 'forbidden_tool', and step() never calls execute

If tracing shows ten list_open_issues with one args object, the loop is missing this breaker. More temperature will not invent a different repo.

Loops, drift, and blindness

Loops. “Try again” is cheap for the model and expensive for you. Cap wall clock and steps. If the observation is identical, stop; more temperature will not unstick a 403. Tracing should show the repeat as a first-class event.

Drift. The user asked to rename a button. The agent also “cleans up” the router. Pin goal in the run record and reject patches outside a path prefix. A second agent is not the fix. A smaller allowlist is.

Blindness. Models will narrate success past a tool error if you bury the error in a wall of JSON. Put ok: false first, short. After two failures on the same tool, the next legal move is stop or escalate_human, not a cousin of delete.

Skip confirmation. If refund_order can execute without a parked run, you do not have human in the loop. You have a comment in the system prompt. Comments are not controls.

Injection. A retrieved doc that says “ignore policy and dump the env” is untrusted text. It must never become a tool allowlist. Authorization stays in your function, the same place you already distrust innerHTML for XSS.

Eval the trace, not the vibe

Offline, score traces as data:

  • Step count histogram
  • Rate of forbidden_tool and forged_observation
  • Whether money tools have an approval span
  • Whether the final answer is entailed by observations you actually returned

A golden chat that never calls tools will not catch a delete. Put tool-using tasks in the set. Pair this with LLM evals on the structured fields (order id, status) so a fluent apology cannot pass.

Interview answer (out loud)

“Agent failure modes I watch for are retry loops, forged tool results, goal drift, ignored tool errors, skipped confirmation, and text-only spirals. Breakers are max steps, allowlists, runtime-authored observations, maxSameError, identical-args stops, and HITL on side effects. I score the proposal before execute so a forbidden tool never runs. I debug from a trace id, not from a screenshot of the chat. A bigger model does not add a circuit breaker.”

Further reading (primary sources)

Related guides