Frontend Beauty
ESC

Type to search the knowledge base.

Coding Agents

A coding agent reads, edits, and runs tests in a repo. Permissions, diffs, and why "it compiles" is not the acceptance test.

intermediate5 min read
  • coding-agents
  • agents
  • devtools

Green tests on a suite the agent deleted are a pass only if you are not looking. Coding agents read, patch, and run commands in a repo. “It compiles” is a compiler’s opinion. Acceptance is the diff, the tests that still exist, and the product checks you did not outsource to the model.

Treat the agent like a junior PR, not like a compiler with vibes. You would not merge a human who removed tests/ and pushed. Do not merge a sampler that did the same with a cheerful summary.

Key takeaways

  • The loop is read, edit, run. Chat-only paste is not a coding agent.
  • Allowlist paths and commands. The model does not get a login shell.
  • Review the diff: scope, secrets, lockfile, snapshots, deleted tests.
  • tsc green is not the product. Frontend still needs a11y and visual checks.
  • Put the test runner in the loop. Do not ask the model whether tests passed.

Coding agents read, edit, and run

A coding agent is a tool loop over a working tree. It lists, greps, reads a slice, applies a patch, runs a pinned test command, and repeats until green or the cap. Products like Claude Code and editor assistants such as GitHub Copilot in VS Code wrap that loop with permissions and UI. Your own version can be a Route Handler plus a sandbox. The shape does not change.

type Patch = { path: string; unifiedDiff: string };

const READ_ROOTS = ['src/', 'tests/', 'package.json', 'tsconfig.json'];
const WRITE_ROOTS = ['src/', 'tests/'];
const TEST_CMD = ['npm', 'test', '--', '--runInBand'];

export async function codingTurn(repo: Repo, goal: string, maxSteps = 12) {
  const beforeTests = await repo.list('tests/');
  let lastLog = '';

  for (let i = 0; i < maxSteps; i++) {
    const action = await model.act({ goal, lastLog, tree: await repo.map() });

    if (action.type === 'read') {
      assertRoot(action.path, READ_ROOTS);
      lastLog = await repo.read(action.path, { limitBytes: 8_000 });
      continue;
    }

    if (action.type === 'patch') {
      for (const p of action.patches as Patch[]) {
        assertRoot(p.path, WRITE_ROOTS);
        await repo.apply(p.unifiedDiff);
      }
      continue;
    }

    if (action.type === 'test') {
      const result = await repo.spawn(TEST_CMD, { timeoutMs: 60_000 });
      lastLog = result.stdout.slice(-4_000);
      if (result.code !== 0) continue;

      const afterTests = await repo.list('tests/');
      if (afterTests.length < beforeTests.length) {
        throw new Error('tests deleted; refusing green');
      }
      return { ok: true as const, diff: await repo.diff() };
    }
  }
  return { ok: false as const, diff: await repo.diff(), lastLog };
}

function assertRoot(path: string, roots: string[]) {
  const n = path.replace(/^\.\//, '');
  if (n.includes('..') || !roots.some((r) => n === r || n.startsWith(r))) {
    throw new Error(`blocked ${path}`);
  }
}

The tests deleted check is the point of the snippet. Typecheck cannot see a missing test.

Keep the working tree dirty in a branch the agent owns. Do not let it commit until a human (or a signed CI policy) says so. A coding agent that git pushes to main is a deploy tool you did not mean to ship. Diff first, PR second, merge last.

Allowlists beat prompt manners

Tool schema design for coding is least privilege:

  • read / grep / apply_patch / run_tests, not bash.
  • If you must have a shell, enumerate npm test, npx tsc --noEmit, npx eslint. No curl, no git push, no rm -rf.
  • Block .env, .pem, credentials.json, and anything your secrets guide already treats as radioactive.
  • Network off by default. Package installs are a human step.

Editor Copilot flows still execute in your workspace. The same rules apply: a suggested terminal command is untrusted text until you run it. HTML the agent writes is untrusted too. If it lands in dangerouslySetInnerHTML, you just invented XSS with extra steps.

Review the diff like a junior PR

The artifact is the patch, not the assistant’s bullet list.

Check Why
File set Drive-by refactors and “while I was here” formatting
Deleted tests / snapshots Cheap green
Lockfile and package.json Unexpected deps, scripts, postinstall
Secrets and URLs Keys in source, webhook to somewhere new
Generated files The agent rewrote dist/ instead of src/
Scope vs ticket Three unrelated components in one PR

You already know this review from humans. Automate the cheap bits in the inner loop (path allowlist, test-tree diff) and in CI eval gates (lint, typecheck, unit, a11y). Leave taste and product intent for a person.

Frontend is not tsc --noEmit

A React change can typecheck and still trap focus, skip a label, or break a visual regression. Put those oracles in the loop when the diff touches UI:

  • axe (or your a11y CI) on the affected route
  • A screenshot test or Chromatic-style check for the component
  • Keyboard path: tab order is not a type

The agent should run your commands, not invent npm test -u. Updating snapshots is a write to the truth. Require a human, the same way you would for a golden master.

Interview answer (out loud)

“A coding agent is a read, edit, run loop over a repo with path and command allowlists. I review the diff like a junior PR: deleted tests, secrets, lockfile, extra files. Compiles and green tests are necessary, not sufficient, especially if the suite shrank. For UI I add a11y and visual checks. The model does not get a free-form shell, and it does not get to tell me the tests passed. The runner does.”

Further reading (primary sources)

Related guides