Frontend Beauty
ESC

Type to search the knowledge base.

PII and Data Handling in LLM Apps

Logs, traces, and prompts store whatever users type. How to redact LLM PII, set retention, and know when a provider trains on traffic.

intermediate6 min read
  • privacy
  • pii
  • security

The eval set that made Slack was also a spreadsheet of customer emails. Someone exported traces, named the file golden.csv, and dropped it in #ai-quality. That is not being data driven. That is an LLM PII leak with a helpful header row.

Users will paste invoices, HR dumps, and “here is the ticket thread.” Your app will then copy that string into the provider request, the trace store, the replay debugger, and the spreadsheet someone uses to argue about prompts. Privacy work is mostly about those copies, not about a sentence in the system prompt that says “do not store secrets.”

Key takeaways

  • Treat prompts, tool results, traces, and eval sets as production data stores. They keep whatever the user typed.
  • Redact on the way out (vendor) and on the way in (your logs). One pass is not enough.
  • “We do not train on your data” is not “we never store this.” Retention and training are different knobs.
  • Zero Data Retention and no-train flags are contract and dashboard config, not prompt text.
  • On the frontend, warn before a paste that looks like a customer export. Keys still stay off the client.

LLM PII is a logging problem first

Map one user message before you buy a classifier.

Copy Who can read it Default fate if you do nothing
Provider request The model vendor, plus whoever can open the vendor dashboard Abuse logs for a retention window; maybe application state for some endpoints
Your trace Anyone with observability access Kept until the index fills the disk
Eval / golden set The whole team, then Slack Lives forever, often unredacted
Support export Whoever got the CSV Email, ticket ids, addresses, the original prompt

The model is not a database, but your pipeline is. Tracing without a redaction step is a second production database with worse access control than Postgres.

Minimize before you generate

Do not send the field if the task does not need it.

  • Support bot: send ticket body and product, not the full CRM row.
  • Docs chatbot: send retrieved chunks, not the user’s entire upload history.
  • Coding agent: send the file the tool read, not .env. Deny that path in the tool schema.

Minimization beats redaction. A regex cannot find a nickname, a voice note, or a screenshot of a passport. If the feature works with an opaque user id, that is the id you log.

import { createHash } from 'node:crypto';

type PromptPayload = {
  userIdHash: string;
  ticketId: string;
  product: string;
  body: string;
};

function hashUserId(userId: string): string {
  return createHash('sha256').update(userId).digest('hex').slice(0, 16);
}

function toPromptPayload(row: {
  userId: string;
  email: string;
  ticketId: string;
  product: string;
  body: string;
}): PromptPayload {
  return {
    userIdHash: hashUserId(row.userId),
    ticketId: row.ticketId,
    product: row.product,
    body: row.body,
  };
}

email never enters the prompt, so it cannot leak through a verbose model, a trace export, or a vendor outage email that quotes the request.

Redact on two edges

Regex is a first pass, not a privacy program. Use it anyway, because the cheap misses are emails, PAN-ish digit runs, and government-id shapes people paste without thinking.

const EMAIL = /\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi;
const SSN = /\b\d{3}-\d{2}-\d{4}\b/g;

export function redactPii(input: string): { text: string; kinds: string[] } {
  const kinds: string[] = [];
  let text = input;

  const apply = (re: RegExp, kind: string, token: string) => {
    const copy = new RegExp(re.source, re.flags);
    if (copy.test(text)) kinds.push(kind);
    text = text.replace(new RegExp(re.source, re.flags), token);
  };

  apply(EMAIL, 'email', '[EMAIL]');
  apply(SSN, 'ssn', '[SSN]');
  return { text, kinds };
}

export function prepareForVendor(userText: string) {
  const { text, kinds } = redactPii(userText);
  return { prompt: text, piiKinds: kinds };
}

export function prepareForTrace(prompt: string, completion: string) {
  return {
    prompt: redactPii(prompt).text,
    completion: redactPii(completion).text,
  };
}

Call prepareForVendor on the server, before the HTTP body leaves your origin. Call prepareForTrace before the span is written. If you only redact logs, the vendor still saw the raw string. If you only redact the vendor call, your replay UI still has the email.

Keep the raw payload only when a reviewer truly needs it (fraud, legal hold), behind an ACL and a TTL.

Training, retention, and ZDR are three questions

People collapse these into “is this private?” Ask them apart.

  1. Does this product train on my traffic? API products at major vendors often default to no, with an explicit opt-in. Consumer chat apps are a different policy. Read the current page for the product you call, not a tweet about the brand.
  2. How long is this stored for abuse monitoring or application state? “No training” still means logs. OpenAI documents abuse-monitoring retention on the order of weeks for many endpoints, and some features keep application state until you delete it. Data controls in the OpenAI platform is the table to re-read when you add an endpoint.
  3. Do we have Zero Data Retention (or the vendor’s equivalent)? That is an approval plus a dashboard switch, and it does not apply to every endpoint. Treat ZDR as product config you test against the vendor’s live table, not a sentence in the system prompt.

Your own retention is still your problem. A 7-day trace index with field redaction will beat an infinite observability project that nobody owns. Put a TTL on eval snapshots too. Yesterday’s production bug should not be a forever-PII archive.

Warn before paste, never put the key in the bundle

Frontend engineers already know the secrets rule: the browser talks to your origin. Same for PII. The client can warn. It cannot be the redaction boundary, because anyone can POST the raw text at your route.

function looksLikeBulkPii(text: string): boolean {
  const emails = text.match(/\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi) ?? [];
  return emails.length >= 3 || text.length > 20_000;
}

export function onComposerPaste(text: string): 'ok' | 'warn' {
  return looksLikeBulkPii(text) ? 'warn' : 'ok';
}

The warn is UX: “This paste looks like a customer list. Continue?” It does not replace server redaction, quota, or auth. Untrusted uploads still need the same distrust you use for XSS and prompt injection.

Footguns

  • Logging the full messages array in a request middleware. That array is the product.
  • Screenshots of traces in tickets. Now the PII is in Jira, too.
  • Using consumer ChatGPT to “clean” a CSV of customers. Different product, different training terms, still a copy you do not control.
  • Trusting the model to mask PII in the reply while you stored the original user turn in cleartext.
  • Shipping a “no-train” claim in marketing without checking which endpoint you actually call.

Guardrails can classify obvious secrets on the way in. They do not replace minimization, TTLs, and “this eval set lives in a bucket with a lock.”

Interview answer (out loud)

“LLM PII is whatever users paste, then whatever we copy: the provider request, traces, and eval sets. I minimize fields before the prompt, redact on the vendor path and again before logging, and I put a TTL on traces. Training-off is not the same as zero retention. ZDR is a contract and a config I would verify per endpoint. The browser can warn on a bulk paste. Redaction and keys stay on the server.”

Further reading (primary sources)

Related guides