Ria feeds two paraphrases and one unrelated sentence through a small model diagram. The resulting coordinate cards move into the map drawer, where the projection suggests a cluster without claiming perfect geometry.
Problem: A vector is only numbers until a model maps text into a space shaped for a task. Ava wants to know what makes nearby points useful.
Resolution: The reader learns that an embedding is a vector representation produced by a model, and that useful similarity is learned and task-dependent rather than guaranteed by proximity alone.
An embedding is a model-produced vector representation. Its geometry can support similarity, but that behavior depends on the model, task, data, and metric.
A model produces the vector
Vectors are ordered lists of numeric coordinates that a system can compare under a chosen rule. Tokenization breaks text into smaller units before a model processes it.
An embedding model maps an input into a fixed-length numeric vector. Text may be tokenized on the way in, but the output is one representation used by downstream systems. The embedding is not an extra coordinate added to a vector. It is the vector representation itself.
The same model normally gives compatible inputs vectors with the same number of dimensions. That fixed shape lets a retrieval system compare a query vector with stored vectors. Compatibility still requires more than matching length: preprocessing, model identity, and model version must agree closely enough for the coordinates to share a meaning.
Mixing vectors from unrelated models is like filing coordinate cards drawn from different maps in one drawer. Both cards may have 768 slots, yet slot 12 in one system has no promised correspondence to slot 12 in the other. Re-embedding the collection is usually safer than assuming the spaces align.
Near is a learned behavior
Training objectives encourage useful patterns in the space. Paraphrases may land near one another, while unrelated inputs may separate. The word "may" matters: models represent different notions of similarity, and a representation that works for one task can perform poorly on another.
A support search might need refund questions near refund policies. A recommendation system might instead need products near items bought by similar users. Those goals can produce different useful neighborhoods from the same text. There is no single context-free arrangement called the true meaning map.
Evaluation must follow the task. Build a set of representative queries, decide which results count as useful, and measure how often the embedding retrieves them under the chosen metric. A few appealing examples cannot establish performance across languages, domains, short labels, long documents, or changing vocabulary.
Projections have limits
Ria's flat drawer is a projection of many coordinates into two visible axes. Projections are useful for explanation and inspection, but they can change apparent distances or hide structure. Production retrieval compares the original vectors, not the drawing.
Individual embedding dimensions are also rarely neat human topics. One coordinate should not be labeled "budget" merely because several budget documents happen to have a high value there. Useful information is commonly distributed across many coordinates and interpreted through comparisons across the whole vector.
The practical record should therefore include the embedding model, version, preprocessing choices, source item, and creation time. Those labels make later migration and re-embedding possible. Without them, a collection can look internally consistent while quietly mixing representations that no longer belong together.
Compatibility is part of the data contract.
Glossary
- Embeddings
- Vector representations produced by a model for inputs such as text, images, or other items. | Office analogy: The model writes a coordinate card for each item.
- Semantic representation
- A task-dependent numeric representation whose geometry may carry useful similarity information. | Example: Two paraphrases may land near each other under one model but not every model or task.
- Embedding model
- The learned function that maps an input to its embedding vector. | Office analogy: The machine that turns an item into its coordinate card.
- Projection
- A lower-dimensional view used to display vectors. It can lose or distort information. | Office analogy: Ria flattens a many-coordinate map into one drawer people can inspect.
Try it yourself
- Teach it back without the map: what does an embedding preserve well enough to compare, and what does it not prove about meaning?
- Compare embeddings only when they come from compatible model versions and preprocessing.
- Treat clusters in a visualization as clues to investigate, not proof that every item shares one meaning.
