How Embeddings Actually Capture Meaning

Part 1 established that embeddings capture meaning as numerical closeness. This part goes one level deeper — what those individual numbers actually represent, and why this real mechanism works as well as it genuinely does.

What a single dimension roughly represents

A real embedding vector has hundreds or thousands of dimensions —
  each one, roughly speaking, captures some real, learned aspect of
  meaning (though rarely anything as cleanly interpretable as a
  single human-nameable concept)

Illustrative simplification (real embeddings are far less clean-cut
  than this, but the underlying idea is genuine):
  dimension 47: correlates loosely with "sentiment" (positive/negative)
  dimension 203: correlates loosely with "formality"
  dimension 891: correlates loosely with "topic: food and beverage"

This simplification genuinely overstates how cleanly interpretable any single real dimension is — in practice, meaning is distributed across many dimensions jointly, not cleanly assigned to individually nameable ones. But the real, underlying principle holds: the model learned, through training, which combinations of dimensions correlate with which real aspects of meaning.

The real, famous vector arithmetic example

A widely cited, real property of well-trained embeddings:
  vector("king") - vector("man") + vector("woman") ≈ vector("queen")

This is a genuinely real, well-documented phenomenon from embedding research — the direction between "king" and "man" in vector space captures something like "royalty," and adding that same real directional shift to "woman" lands near "queen." This isn't hardcoded or manually engineered — it emerges from the real training process, a genuine, concrete demonstration that these vectors capture actual semantic relationships, not just superficial word similarity.

Why different embedding models produce genuinely different, incompatible vector spaces

Model A's embedding for "coffee": [0.12, -0.44, 0.91, ...]
Model B's embedding for "coffee": [0.67, 0.03, -0.28, ...]

These real vectors are NOT directly comparable to each other, even for the exact same real word — each embedding model learns its own, real, internally-consistent vector space during training, with no guaranteed relationship to a different model's vector space. This is a genuinely important, practical constraint: a real application must use the same embedding model to generate every vector it ever compares — mixing vectors from two different models produces meaningless, incorrect similarity scores.

A real, practical consequence for this series' RAG project

Correct: embed Bright Leaf Coffee's real FAQ documents with Model A,
  then embed every real incoming customer question with that SAME
  Model A, for every comparison
Incorrect: embedding documents with one real model, then later
  switching to a different embedding model for new questions without
  re-embedding the existing documents too

This directly sets up a real, practical requirement for part 6's actual RAG build — the embedding model choice has to stay consistent across an entire real system's lifetime, or every existing document needs real, complete re-embedding if it ever changes.

Why embeddings generalize better than exact keyword rules

A hand-written synonym list: "stale" = "old" = "not fresh" — genuinely
  requires a real person to anticipate and encode every real variation
A trained embedding model: learns these real relationships (and
  countless others never explicitly listed) directly from patterns
  in a massive amount of real training text

This is the real, honest reason embeddings represent a genuine improvement over the AI Projects series' own keyword-matching approach — rather than a person manually anticipating every real way a customer might phrase a question, the embedding model has already, genuinely learned a vast range of real semantic relationships from its training data.

Next: vector databases explained — the real, practical infrastructure for storing and searching millions of these embedding vectors efficiently.