Frontend Beauty
ESC

Type to search the knowledge base.

What Is AI Engineering?

AI engineering is shipping LLM products: prompts, retrieval, tools, evals, and cost control. How it differs from ML research and from wrapping a chat API.

beginner5 min read
  • ai-engineering
  • llms
  • career

You can wrap a chat API in a weekend. The hangover shows up on Monday: a user pasted an API key into the box, the model invented a refund, and the bill jumped because a retry loop kept “trying again.” AI engineering is the work of making that wrapper into a product: typed interfaces, retrieval, tools with authorization, evals, traces, and a cost ceiling.

If you already ship React apps, this is closer to backend-for-frontend work than to training a model. You still need to know what the model is (a next-token sampler with a context window). You do not need a research internship to ship a cited docs chatbot.

Key takeaways

  • AI engineering is product software around models: prompts, data, tools, evals, and operations.
  • The model is a probabilistic HTTP API. It has no memory, no permissions, and no idea what “true” means.
  • Most failures are app failures: bad retrieval, missing auth on tools, no evals, unbounded loops.
  • Frontend engineers already have the right instincts: types, abort, loading states, XSS-shaped distrust of untrusted text.

What AI engineering is, in one sentence

AI engineering is building and operating applications that call large language models, then proving those applications do the right thing often enough to ship.

That includes:

Layer What you own
Interface Chat UI, streaming, abort, conversation ids
Orchestration Prompts, tools, retrieval, agent loops, max steps
Quality Golden sets, judges, CI gates, production traces
Safety Auth on tools, prompt-injection assumptions, PII
Ops Routing, cache, retries, cost, latency (TTFT)

Training a new base model is not this job. Fine-tuning sometimes is, and it is usually the last lever after prompts, tools, and retrieval. See fine-tuning: when it helps once you have an eval set that a prompt cannot move.

What it is not

Not ML research. You will not derive attention from scratch to ship a support bot. You will need a working picture of tokens and context windows, embeddings, and why temperature is a sampler, not a “creativity slider.”

Not “prompting.” A prompt is a function spec. If changing a comma flips production behavior, you are missing tests, not adjectives. Prompt engineering fundamentals treats the text as code.

Not a chat widget. If the model cannot call tools or read your documents, you have a text box in front of a public model. That can be the right product. Call it a chatbot. Agents start when the model can change something outside the string, inside a loop you cap.

A request is just fetch with worse guarantees

This is the mental model that maps onto frontend work:

type ChatMessage =
  | { role: 'system'; content: string }
  | { role: 'user'; content: string }
  | { role: 'assistant'; content: string }
  | { role: 'tool'; toolCallId: string; content: string };

type CompletionRequest = {
  model: string;
  messages: ChatMessage[];
  tools?: ToolSchema[];
  maxTokens: number;
};

// The response is sampled. Same request, different Tuesday.
type CompletionResponse = {
  message: ChatMessage;
  usage: { inputTokens: number; outputTokens: number };
};

Treat it like any other dependency:

  • Timeouts and abort. Users mash Stop. Use AbortController the same way you would for search. See AbortController.
  • No secrets in the bundle. The browser talks to your origin. Keys stay on the server. See secrets in frontend bundles and client vs server LLM calls.
  • Untrusted text is untrusted text. Retrieved HTML and uploaded PDFs can contain instructions. That is prompt injection, which rhymes with XSS.
  • You own side effects. If the model emits refund_order, your code still checks the user owns that order. The JSON is a proposal.
async function runRefundTool(
  user: { id: string },
  args: { orderId: string; amountCents: number },
) {
  const order = await db.orders.findOwned(user.id, args.orderId);
  if (!order) throw new Error('not found');
  if (args.amountCents > order.paidCents) throw new Error('amount');
  // Human approval belongs here for irreversible money movement.
  return refundProvider.credit(order.id, args.amountCents);
}

The model never talks to Stripe. Your process does, after policy.

The four product loops you will actually build

Almost every LLM feature is one of these, or a combination.

  1. Generate. Draft, summarize, extract into a schema. Start here. Add structured outputs before you write a JSON repair library.
  2. Retrieve then generate (RAG). Search your corpus, pack chunks into the prompt, ask the model to answer from that. The model only knows what you stuffed in. RAG mental model.
  3. Tool loop (agents). Think, call a tool, observe, repeat, stop. Cap steps. The think, act, observe loop.
  4. Evaluate. A golden set you run on every prompt change. Without it you are vibe-shipping. LLM evals.

If you are a frontend engineer picking a first project, build (2) on this site’s MDX: ingest pages, retrieve, stream an answer with citations. The walkthrough is building a docs RAG chatbot.

How you know you are doing the job

A wrapper becomes engineering when you can answer:

  • What did we send? Prompt version, retrieved chunk ids, tool schemas.
  • What happened? A trace id with token counts and latencies. Tracing.
  • Did it get worse? A gate on a golden set in CI. Eval gates.
  • What does it cost on p95? Cache, routing, max tokens. Cost.
  • What can it not do? Tool allowlists, approval for side effects, refusal when retrieval is empty.

None of those require a GPU in your closet. They require the same production habits you already use for payments and search.

Interview answer (out loud)

“AI engineering is shipping software that calls LLMs: the prompt, the data you retrieve, the tools you allow, and the evals that catch regressions. The model is a sampler with a context window, not a database and not an authorization layer. I would start with a typed server route, retrieval with citations, traces, and a small golden set. I would not start by fine-tuning.”

Further reading (primary sources)

Related guides