Module 7 — Choosing a Vector Database

Now the practical question: which one do I actually use? There are many options and the marketing is loud. This module cuts through it with a clear mental map, honest trade-offs, and a decision flow.

First, a crucial distinction: library vs. database vs. extension

These are three different things people lump together:

Vector libraries (FAISS, Annoy, hnswlib, ScaNN) — code you embed in your own app. They do indexing + search in memory and that’s it. No server, no storage durability, no metadata filtering to speak of, no clustering, no auth. You handle persistence, updates, scaling. Brilliant for prototypes, research, and when you want maximum control. FAISS (from Meta) is the gold standard and powers many products under the hood.
Vector databases (Milvus, Qdrant, Weaviate, Pinecone, Vespa, Chroma) — full systems: a server you talk to over the network, durable storage, metadata + filtering, CRUD, sharding/replication, auth, monitoring. Use these when vectors are central to your product and you need real operations.
Existing databases with vector support (Postgres + pgvector, Elasticsearch/OpenSearch, Redis, MongoDB Atlas, ClickHouse, SQLite + sqlite-vec) — your familiar database gains a vector index. Huge advantage: vectors live next to your existing data, transactions, and filters; one system to operate. Often the smartest first choice because it adds the least new complexity.

The most underrated advice in this whole course: most teams should start with pgvector (or whatever database they already run) and only graduate to a dedicated vector database when they actually hit its limits. Adding a whole new distributed system is a big operational cost; don’t pay it before you must.

The contenders, honestly

pgvector (Postgres extension)

What: Adds a vector column type and HNSW/IVFFlat indexes to Postgres.
Loves: Teams already on Postgres; combining vectors with SQL joins, filters, transactions; small-to-mid scale (up to ~10M–50M vectors comfortably, more with tuning).
Watch: Not built for billions; scaling is Postgres scaling. Newer extensions (pgvectorscale) push limits further.
Verdict: The default starting point for most apps. Boring in the best way.

Chroma

What: Lightweight, developer-friendly, “import chromadb and go.” Embeds in-process or runs as a server.
Loves: Prototypes, local dev, small RAG apps, notebooks.
Watch: Less proven at very large scale / heavy production load.
Verdict: Fastest path to a working RAG demo.

Qdrant

What: Rust-based, fast, strong filtering, great quantization support, good single-node and clustered modes. Open source + managed cloud.
Loves: Production RAG with rich metadata filtering; cost-conscious teams wanting strong performance; on-prem.
Verdict: A top pick for serious open-source production use. Excellent default for a dedicated DB.

Weaviate

What: Open source, has built-in modules to generate embeddings, hybrid search, and even RAG steps. GraphQL API.
Loves: Teams wanting batteries-included semantic search with hybrid out of the box.
Verdict: Strong all-rounder, especially if you like its integrated modules.

Milvus

What: The heavyweight for massive scale (billions of vectors). Cloud-native, separates storage/compute, many index types, GPU support. Zilliz is the managed version.
Loves: Very large datasets, high throughput, teams with platform/ops capacity.
Watch: More moving parts; heavier to operate yourself.
Verdict: When you genuinely need billion-scale and a real cluster.

Pinecone

What: Fully managed, serverless, no ops. You call an API; they run everything.
Loves: Teams that want zero infrastructure work and predictable scaling; fast to production.
Watch: Proprietary/managed-only; cost at scale; data leaves your infra (matters for some compliance).
Verdict: Best when you’d rather pay to never think about ops.

Elasticsearch / OpenSearch

What: Mature search engines that added dense-vector (HNSW) search alongside their world-class keyword search.
Loves: Teams already running them; hybrid search (keyword + vector) where their BM25 is excellent.
Verdict: Great if you’re already invested or need best-in-class lexical + vector together.

Redis (Redis Stack / RediSearch)

What: In-memory vector search bolted onto Redis.
Loves: Ultra-low latency, caching layers, you already run Redis.
Verdict: Speed-first, smaller-scale or hot-set use cases.

Vespa

What: Yahoo’s heavyweight engine for search + ranking + ML, very powerful for complex relevance and large scale.
Verdict: Powerful but steeper learning curve; shines on complex ranking needs.

FAISS (library, not a server)

What: Meta’s industry-standard ANN library. Every index type, CPU/GPU, battle-tested.
Verdict: Use directly for research/control, or know that it’s the engine inside many of the above.

Others worth knowing

MongoDB Atlas Vector Search, SQLite + sqlite-vec (great for edge/local apps), LanceDB (embedded, columnar, disk-based, growing fast), Vald, Marqo, pgvectorscale, Turbopuffer (serverless, object-storage-backed).

The decision flow

Do you already run Postgres / Elasticsearch / Redis / Mongo?
   └─ Yes, and < ~10M vectors → use its vector feature (pgvector etc.). Stop. ✅
Just prototyping / a notebook / local app?
   └─ Chroma, LanceDB, or sqlite-vec. ✅
Need a production dedicated DB, open source, strong filtering?
   └─ Qdrant or Weaviate. ✅
Want zero ops, fully managed?
   └─ Pinecone (or Zilliz / Qdrant Cloud / managed Weaviate). ✅
Billions of vectors, high throughput, have platform team?
   └─ Milvus / Vespa. ✅
Need best-in-class keyword + vector hybrid?
   └─ Elasticsearch / OpenSearch / Vespa. ✅

The evaluation checklist (use this when comparing for real)

When you shortlist, score each option on what your app needs:

Scale: vectors now and in 2 years; queries/second; insert rate.
Latency target: p95/p99 you must hit (Module 10).
Recall target: how exact must results be? (Module 11)
Filtering: how rich are your metadata filters, and how selective? (Module 9) — a make-or-break differentiator.
Hybrid search: do you need keyword + vector together? (Module 9)
Updates: static dataset, or constant inserts/updates/deletes?
Multi-tenancy: many isolated customers in one system? Per-tenant filtering/namespaces?
Ops model: self-host vs managed; team’s appetite for running a cluster.
Cost: RAM is the dominant cost driver; quantization and disk-based indexes change the math a lot.
Compliance / data residency: can data leave your infra? On-prem required?
Ecosystem: SDKs, LangChain/LlamaIndex integration, community, docs, longevity.
Consistency / durability: what happens on crash; replication; backups.

A reusable mental model

Almost every product is some combination of: (an ANN index from Module 5) + (quantization from Module 6) + (a storage engine) + (a metadata filter engine) + (a distributed systems layer) + (an API). Once you understand Modules 5–6, you can evaluate any new database by asking “which index, which compression, how does it filter, how does it scale?” The names change; the fundamentals don’t.

Hands-on: the same task in three tools

pgvector (SQL):

CREATE EXTENSION vector;
CREATE TABLE docs (id bigserial PRIMARY KEY, content text, embedding vector(384));
CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);
-- search: <=> is cosine distance (smaller = closer)
SELECT content FROM docs ORDER BY embedding <=> '[...]'::vector LIMIT 5;

Chroma (Python):

import chromadb
client = chromadb.Client()
col = client.create_collection("docs")
col.add(ids=["1","2"], documents=["the cat sat","revenue grew"],
        embeddings=[[...],[...]])
col.query(query_embeddings=[[...]], n_results=5)

Qdrant (Python):

from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance, PointStruct
c = QdrantClient(":memory:")
c.create_collection("docs", VectorParams(size=384, distance=Distance.COSINE))
c.upsert("docs", [PointStruct(id=1, vector=[...], payload={"text":"the cat sat"})])
c.search("docs", query_vector=[...], limit=5)

Notice the shape is identical everywhere: create a collection with a dimension + metric, upsert vectors with metadata, query with a vector for top-k. Learn the pattern once; every tool is a variation.

Gotchas

Choosing the trendiest, not the simplest. Most teams don’t need a distributed vector database. pgvector or Chroma ships your product faster.
Ignoring filtering quality. Two databases with identical pure-vector speed can differ wildly once you add “WHERE tenant = X.” Test filtered queries, not just raw ones.
Benchmarking with random vectors. Synthetic data hides real recall behavior. Benchmark with your embeddings and your query patterns.
Forgetting the total cost of ownership. A “free” open-source DB you must staff, scale, back up, and monitor may cost more than a managed one.
Lock-in blindness. Managed/proprietary tools are convenient but moving 500M vectors out later is painful. Keep your raw text + embedding pipeline portable.

Check yourself

What’s the difference between a vector library, a vector database, and a database with vector support?
Why is pgvector often the smartest first choice?
Name three checklist items that distinguish databases beyond raw search speed.
Reduce any vector database to its component parts (the mental model).

Answers: (1) A library (FAISS) is in-process index+search only — you handle storage/scaling; a vector database (Qdrant, Milvus) is a full server with durability, filtering, CRUD, and clustering; a database-with-vectors (pgvector) adds vector indexing to a system you already run. (2) It keeps vectors beside your existing data/filters/transactions in one system you already operate, adding the least new complexity, and handles up to tens of millions of vectors. (3) Any three of: filtering quality, hybrid search, update patterns, multi-tenancy, ops model, cost, compliance, ecosystem, durability. (4) ANN index + quantization + storage engine + filter engine + distributed layer + API.

➡️ Next: 08-building-rag.md — build a real retrieval app end to end.