Frontend Beauty
ESC

Type to search the knowledge base.

Vector Databases Explained

A vector database stores embeddings plus metadata and runs ANN search. What you actually buy vs pgvector in Postgres, and the ops you will own.

intermediate7 min read
  • vector-db
  • infrastructure
  • rag

You do not have a “memory layer.” You have an index that can go stale, disagree with the source of truth, and keep serving last week’s handbook after legal updated the PDF.

A vector database is storage plus nearest-neighbor search over embedding vectors: an id, a float array, metadata, and an index (exact or approximate) so you can ask for neighbors with filters. Postgres with pgvector is a vector database in that sense. So is a dedicated product. The category is “search over vectors,” not a soul, and not a substitute for the document store you already have.

Key takeaways

  • The product is ANN search plus filters, not “memory.”
  • You still own upsert, deletes, re-embeds, and tenant isolation.
  • Postgres plus pgvector is enough for many app-sized corpora.
  • Dedicated stores buy ops, scale, and hybrid features. They do not remove staleness.
  • Namespaces and metadata filters are how you avoid mixing customers, not prompt wording.

What a vector database actually stores

Each row is roughly:

type VectorRow = {
  id: string;
  values: number[]; // embedding; length is model-specific and must match the index
  metadata: {
    tenantId: string;
    url: string;
    headingPath: string[];
    updatedAt: string;
  };
};

type Query = {
  vector: number[];
  topK: number;
  filter: { tenantId: { $eq: string } };
  namespace?: string;
};

Vendors (Pinecone, Qdrant, Weaviate, and the rest) sell this inventory with different APIs. If a pitch deck says “long-term memory for agents,” translate it to this row type.

Keep chunk text beside the vector (same row or object storage keyed by id). Search returns ids. You pack text. Vectors alone cannot be cited.

How ANN search actually walks

Embeddings and similarity search is k-NN versus ANN as a product decision. Here is the index you are buying.

Brute force scores the query against every row. Exact for the metric. Fine until n * dimension no longer fits the latency budget.

HNSW (hierarchical navigable small world), from Malkov and Yashunin, is a layered proximity graph. Layer 0 holds every node with short edges. Higher layers are sparse, with longer edges; a node’s max layer is drawn like a skip list. Insert greedy-searches from the top, then connects the new node to up to M neighbors (m in pgvector). ef_construction is the candidate-list size while choosing those neighbors: higher means a better graph, a slower build, more RAM. Search starts at the top-layer entry point, walks to a local minimum, and drops a layer. On layer 0 it expands efSearch / ef_search candidates and returns the closest k. The walk is approximate because it can miss a better basin; raising efSearch explores more before stopping.

IVF / IVFFlat clusters vectors with k-means into lists. A query scores only nprobe / probes lists whose centroids are closest. Build needs data (centroids are trained). Inserts after build can land in the wrong list; recall drifts until you rebuild.

Index How it finds neighbors You tune Breaks when
Brute force / seq scan Score every row Nothing Millions of rows, tight p95
HNSW Greedy walk on a layered graph m, ef_construction, ef_search Aggressive filters, uncompacted deletes
IVF / IVFFlat Search nprobe clusters lists, nprobe Empty-table build, distribution shift after inserts
Filter then scan SQL/WHERE first, brute the survivors The filter The filter is not selective

Filtered ANN is the p95 killer. If 99% of neighbors fail tenantId = acme, the walk is in the wrong city. Engines then iterative-scan or brute the survivors. Plot recall@k versus ef_search / nprobe on a labeled set before you ship.

Deletes often mark a node and leave it in the graph. Search still hops through tombstones until compaction. Vector index updates is the write-side contract. Tens of thousands of chunks: brute force in Postgres may beat a poorly tuned ANN. Measure.

Postgres with pgvector until it is not

pgvector adds a vector column, distance operators (<-> L2, <=> cosine, <#> inner product), and optional IVFFlat or HNSW indexes. You keep one database, one backup story, one transaction with the rest of the row.

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);
SET hnsw.ef_search = 40;
You might stay on Postgres when You might leave when
App-sized corpus, one region, team already runs PG Hundreds of millions of vectors, multi-region, bursty QPS
Filters are ordinary SQL WHERE You need sparse+dense hybrid as a first-class query
Embeddings update in the same transaction as the article You want a managed ANN service and will pay for it
You can accept vacuum, maintenance_work_mem during HNSW build, and recall plots You do not want to own index build time or compaction

Leaving is not a moral upgrade. It is another vendor, another consistency story, another bill. Stay versus leave on p95, recall, and ops time, not a tweet. Stay and you own WAL growth, VACUUM after bulk deletes, HNSW build RAM, and iterative scans when filters are selective. Leave and you own a second ingest path and dual backups.

Ops you will own either way

Upsert versus insert. Re-embedding a doc must replace the old vectors for that url, not append a second copy. Insert-only indexes accumulate ghosts.

Deletes. Legal said take it down. Until you delete and compact, the graph still hops through that id.

Model identity. Dimension and metric are index-level. Changing embedding models means a new index and a backfill. Half-migrated namespaces rank garbage.

Filters and namespaces. Tenant id on every row, filter on every query. Namespaces (collections, schemas) are extra isolation, not a substitute for a WHERE. See metadata filters in retrieval.

Staleness. The CMS is source of truth. The vector index is a derived view. If your ingest job dies at 2am, chat is wrong until it runs. Treat it like a search index, which it is.

Split-brain. Dual-writing Postgres and a hosted vector DB without a single ingest worker is how one side has the 2026 policy and the other does not.

Delete-then-embed-then-upsert is one unit of work for one worker per article id. On pgvector, one SQL transaction. On a remote ANN API, still serialize per id so two jobs cannot interleave.

async function upsertArticle(article: {
  id: string;
  tenantId: string;
  url: string;
  chunks: { text: string; headingPath: string[] }[];
  embed: (text: string) => Promise<number[]>;
  index: {
    deleteByArticleId: (id: string) => Promise<void>;
    upsert: (rows: VectorRow[]) => Promise<void>;
  };
}) {
  // Single ingest worker. Remote ANN APIs are not a Postgres transaction;
  // still do not let a second job for this article id start until this returns.
  await article.index.deleteByArticleId(article.id);
  const rows: VectorRow[] = [];
  for (const [i, chunk] of article.chunks.entries()) {
    rows.push({
      id: `${article.id}:${i}`,
      values: await article.embed(chunk.text),
      metadata: {
        tenantId: article.tenantId,
        url: article.url,
        headingPath: chunk.headingPath,
        updatedAt: new Date().toISOString(),
      },
    });
  }
  await article.index.upsert(rows);
}

Failure: deleteByArticleId skipped because “upsert will overwrite.” It will not, if ids changed after you altered the splitter.

A stale index is not a cache-control problem. Vector index updates is the pointer for tombstones, rebuilds, and aliases.

Interview answer (out loud)

“A vector database stores embeddings, ids, and metadata, and runs ANN search with filters. HNSW is a layered graph you tune with efSearch; IVF searches nprobe clusters. I would start with pgvector in Postgres if the corpus is app-sized and we already operate Postgres. I would move to a dedicated store when latency, scale, or hybrid features demand it. Either way I own upsert-after-delete, re-embeds on model change, compaction after deletes, and tenant filters. It is a derived search index, not memory.”

Further reading (primary sources)

Related guides