← Back to concept explainers

Concept Fable

Word embeddings

Conclusion first:Word Embedding is a technique for representing words or tokens as vectors, enabling models to capture statistical and semantic relationships between words through spatial position. Similar words are usually closer in vector space, and relationship patterns can also be used by the model. It is the foundation of early NLP, semantic retrieval, and modern embedding ideas, but word vectors inherit bias, ambiguity, and missing context from the training corpus.

Word Embedding · Embeddings

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Words moved into the map

White Paper City's dictionary was thick, with every word living on its own page. To look up "doctor" you had to flip to the medical radical section; to look up "hospital" you also had to flip there; to look up "nurse" it was somewhere else. They clearly often appeared together, yet acted like strangers.

The children in the schoolhouse memorized words quickly, but when asked whether "cat" and "tiger" were closer, or whether "cat" and "ledger" were farther apart, the dictionary could not give a distance. The teacher first solved it with numbering: the first page is one, the second page is two. Numbering made storage convenient, but absurdly made two unrelated words on adjacent pages look very close.

A mapmaker began arranging homes for words according to usage. Words that often appeared with patients, medicine, and surgery lived on the same block; words that often appeared with navigation, wind, and ships lived by the harbor. For the first time, words had direction and distance. Finding similar expressions, making recommendations, and understanding queries all became smoother than flipping through paper pages.

But the map also inherited the old city's problems. Old newspapers lumped certain occupations together with certain types of people, and the map drew that bias into the streets; a word with two meanings could also be squeezed into an awkward position.

Later, the city invented maps that change with sentences. The same word can temporarily move to a more suitable neighborhood in different contexts. The gentleman later had words move into a map. Words that often appear together and often substitute for each other in similar sentences live close by; words with little relation live far apart.

The children were excited the first time they saw the map: "doctor" was near "nurse," "tiger" was near "cat" but far from "ledger"; "apple" still stood between fruit and the carpenter's shop, reminding everyone that words can have multiple identities.

This small incident made everyone stop and review. They did not rush to swap in a more impressive-sounding explanation, but instead wrote down clearly, one by one, the entry points that went wrong, the rules that were misunderstood, and the boundaries that needed to be preserved.

This map can't replace dictionary definitions, but it gives words distance from one another. Many judgments no longer rely solely on page numbers and literal order. The old gentleman still keeps the first word map, because it made everyone understand: words aren't just symbols—they can also become spatial positions a machine can compute.

Reveal

This story is about: word embeddings

Word Embedding is a technique for representing words or tokens as vectors, enabling models to capture statistical and semantic relationships between words through spatial position. Similar words are usually closer in vector space, and relationship patterns can also be used by the model. It is the foundation of early NLP, semantic retrieval, and modern embedding ideas, but word vectors inherit bias, ambiguity, and missing context from the training corpus.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Word map: vector space
  • Adjacent stalls: words similar in meaning or usage
  • Dictionary order only: discrete symbols cannot express similarity relationships
  • The streets of old corpora: statistical co-occurrence in training data
  • Misplaced words: ambiguity and bias
  • The Later Context Map: From Static Word Vectors to Contextual Representations

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

embeddingword2vecGloVetoken embedding
京ICP备2026014272号