Word Embedding
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A representation of a word as a list of numbers, positioned so that words with similar meanings sit close together – the way language models encode meaning.
What is a Word Embedding?
A word embedding represents a word as a vector, a list of numbers positioned so that words with similar meanings sit near each other. Distance and direction in that space carry semantic information.
Models compute on numbers, not words. Older schemes like one-hot encoding gave every word its own isolated slot, so “cat” and “dog” looked exactly as unrelated as “cat” and “carburetor”. Embeddings replace that with a geometry of meaning.
How Word Embeddings Work
Embeddings are learned from context. Words that appear in similar surroundings across a large text corpus end up with similar vectors – nobody defines what “cat” means; its usage does.
There are two routes to an embedding. Algorithms like Word2Vec, published by Google researchers in 2013, train embeddings as a standalone product that any model reuses. Alternatively, a network learns them on the job, inside its own embedding layer.
Classic embeddings are static: one vector per word, whatever the sentence. Transformer models go further and compute context-dependent vectors, so “bank” gets different representations in “river bank” and “bank account”.
Word Embedding vs Embedding Layer
The word embedding is the representation; the embedding layer is the network component that stores and learns it. One is data, the other is machinery.
| Word embedding | Embedding layer | |
|---|---|---|
| What it is | The representation: one vector per word | A trainable lookup table inside a neural network |
| Where it lives | Anywhere vectors are stored; reusable across models | Inside one specific model |
| How it is produced | By Word2Vec, an embedding layer, or another training method | Learned with the rest of the network via backpropagation |
Example of a Word Embedding
Take trained embeddings and do arithmetic on them: start with the vector for “king”, subtract “man”, add “woman”. The nearest word to the result is “queen”.
Nobody taught the model about royalty or gender. The relationship emerged from word co-occurrence statistics and became a direction in the space.
The same geometry has a governance side: researchers showed that embeddings trained on web text reproduce stereotypes found there, such as gendered associations with occupations. Testing for bias therefore extends to the embeddings themselves, not just a model’s final outputs.
Related AI terms: Word2Vec · Embedding Layer · Tokenization · Latent Space · Natural Language Processing
Did you like the Word Embedding gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About