Module 3 — Vectors and Similarity Metrics (Gentle Math)

You can’t be an expert without comfort here, but the math is friendlier than it looks. We’ll keep it concrete.

What is a vector, really?

A vector is just an ordered list of numbers: [0.2, -0.5, 0.9]. Two ways to picture it:

As a point in space. [0.2, -0.5, 0.9] is a location in 3D space. An embedding with 768 numbers is a point in 768-dimensional space. You can’t visualize 768D — nobody can — but every rule from 2D and 3D still applies mathematically.
As an arrow from the origin (the all-zeros point) to that location. The arrow has a direction and a length (magnitude). This view matters because some similarity metrics care about direction, others about exact position.

Dimension = how many numbers are in the list. “High-dimensional” just means “a long list.” Vector search lives in high dimensions (hundreds to thousands), which is where weird things happen — see “the curse of dimensionality” below.

Measuring “closeness”: the three metrics you must know

When we say two vectors are “similar,” we need a number for it. Three metrics dominate.

1. Euclidean distance (L2) — “as the crow flies”

The straight-line distance between two points, exactly like a ruler. In 2D:

distance = √((x₂-x₁)² + (y₂-y₁)²)

In N dimensions you just add more squared differences under the root. Smaller = more similar (distance 0 means identical). It cares about absolute position, so vector length matters.

2. Cosine similarity — “do they point the same way?”

Measures the angle between two arrows, ignoring their length. It ranges from +1 (pointing the same direction = very similar) to 0 (perpendicular = unrelated) to −1 (opposite = opposite meaning).

cosine(A, B) = (A · B) / (|A| × |B|)

where A · B is the dot product (multiply matching numbers, sum them up) and |A| is the length of A.

Why it’s the default for text: Meaning lives in direction, not magnitude. A long document and a short one about the same topic point the same way even if one arrow is longer. Cosine ignores that length difference and just asks “same direction?” That’s usually what you want for semantic search.

3. Dot product (inner product) — “direction and length together”

A · B = a₁b₁ + a₂b₂ + ... + aₙbₙ

It rewards vectors that both point the same way and are long. Some models (and recommendation systems) deliberately use length to encode “importance” or “popularity,” so dot product is the right metric for them.

The relationship that ties it together

If your vectors are L2-normalized (rescaled to length 1), then cosine similarity = dot product, and ranking by dot product = ranking by Euclidean distance.

This is why so many systems normalize their embeddings: you get cosine’s “direction-only” behavior while using the cheaper dot-product computation. Many databases let you pick cosine, dot, or l2 as the metric — and you must pick the one your embedding model was trained for (check the model card; most modern text models want cosine / normalized dot product).

A worked example by hand

A = [1, 0]      (pointing right)
B = [0, 1]      (pointing up)
C = [2, 0]      (pointing right, but longer)

Euclidean(A, B) = √(1²+1²) = √2 ≈ 1.41
Cosine(A, B) = (1·0 + 0·1)/(1×1) = 0 → perpendicular, unrelated.
Cosine(A, C) = (1·2 + 0·0)/(1×2) = 1 → same direction, “identical meaning.”
Dot(A, C) = 1·2 + 0·0 = 2 → high, rewarded for both direction and C’s extra length.
Euclidean(A, C) = √(1²+0²) = 1 → not zero, because C is farther out even though same direction.

Notice how A and C are “identical” by cosine but “different” by Euclidean. Choosing the metric changes the answer. That’s the lesson.

Which metric should I use?

Text semantic search / RAG: cosine (or dot product on normalized vectors). Default choice.
Recommendations where popularity matters: dot product (length encodes importance).
Computer vision / clustering / when magnitude is meaningful: Euclidean is common.
When in doubt: read your embedding model’s card. It will tell you. Using the wrong metric is a top-3 cause of “my search results are bad.”

The curse of dimensionality (why high-D is strange)

In everyday 2D/3D space, “nearest neighbor” is intuitive. In hundreds of dimensions, geometry gets weird:

Distances bunch up. The nearest and farthest points become almost equally far away. The contrast between “close” and “far” shrinks.
Everything is “far.” Volume explodes with dimensions, so points spread out and become sparse — your data is mostly empty space.
Intuition fails. Random vectors are almost always nearly perpendicular to each other.

Why you should care: This is exactly why exact nearest-neighbor search is hard at scale and why we need clever approximate indexes (Module 5). It’s also why good embeddings (which concentrate meaning into useful directions) matter so much — they fight back against the curse by giving structure to the space.

Normalization, in practice

To L2-normalize a vector, divide it by its length:

import numpy as np
def l2_normalize(v):
    return v / np.linalg.norm(v)

After this, every vector has length 1 and lives on the surface of a unit hypersphere. Most text-embedding pipelines normalize once at insertion and once per query, then use dot product. Many libraries do this for you (normalize_embeddings=True).

Other metrics you’ll occasionally meet

Manhattan / L1 — distance along grid lines (sum of absolute differences). Sometimes more robust in high dimensions.
Hamming distance — number of differing bits, used for binary (1-bit) embeddings; extremely fast (Module 6).
Jaccard — overlap of sets, used for sparse/keyword vectors.

Gotchas

Metric mismatch. Indexing with cosine but querying with L2 (or vice versa) silently ruins ranking. Set the metric once, consistently.
Forgetting to normalize when the metric assumes it. Cosine results look “almost right but subtly off.”
Comparing scores across metrics. A cosine of 0.8 and an L2 of 0.8 mean completely different things. Don’t threshold blindly — calibrate on real data.
Trusting raw similarity scores as probabilities. A cosine of 0.7 is not “70% relevant.” Scores are only meaningful relative to each other within the same model/metric.

Check yourself

What’s the difference between what cosine similarity and Euclidean distance “care about”?
When are cosine and dot product equivalent?
Why is high-dimensional nearest-neighbor search hard (one sentence)?
Your embedding model card says “use cosine similarity, vectors are normalized.” Which database metric do you pick, and do you need to normalize again?

Answers: (1) Cosine cares only about direction (angle); Euclidean cares about absolute position including length. (2) When the vectors are L2-normalized to length 1. (3) The curse of dimensionality makes near and far distances bunch together, so “nearest” loses contrast. (4) Pick cosine (or dot product since they’re equivalent on normalized vectors); the model already normalizes, but normalizing again is harmless and often expected.

➡️ Next: 04-similarity-search.md — doing the search exactly, and why it falls over at scale.