BERT
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A language model from Google that reads text in both directions at once to understand meaning, used in search, classification, and other language tasks.
What is BERT (Bidirectional Encoder Representations from Transformers)?
BERT (Bidirectional Encoder Representations from Transformers) is a language model from Google that reads an entire sentence in both directions at once to work out what each word means in context.
Earlier language models read text left to right, so a word was interpreted only from what preceded it. BERT sees every word’s full surroundings, which matters whenever the words that follow a term change its meaning. Its job is understanding text, not generating it.
How BERT Works
Google researchers introduced BERT in 2018 with a training method called masked language modeling. Random words in a sentence are hidden, and the model must predict them from the words on both sides. Getting good at this forces it to learn how words relate to one another.
Architecturally, BERT is the encoder half of the transformer: it converts text into rich numerical representations rather than producing new text.
Once pre-trained, BERT adapts cheaply. Fine-tuning on a small labeled dataset turns the same base model into a question answerer, a sentiment analysis tool, or a named entity extractor.
BERT vs GPT
The practical difference: BERT reads, GPT writes. BERT is a transformer encoder that turns text into representations for understanding tasks such as search and classification. GPT is a transformer decoder trained to predict the next word, so it generates text; BERT cannot.
| Criterion | BERT | GPT |
|---|---|---|
| Architecture | Transformer encoder | Transformer decoder |
| Reads text | Both directions at once | Left to right, token by token |
| Training objective | Predict masked-out words | Predict the next token |
| Built for | Understanding and classifying text | Generating text |
Example of BERT
Google announced in 2019 that BERT helps interpret Search queries. Take the query “can you pick up medicine for someone else at the pharmacy.” A keyword system latches onto “medicine” and “pharmacy” and returns generic pharmacy pages. The three small words “for someone else” carry the actual question.
Because BERT reads the whole query bidirectionally, it registers that “for someone else” modifies the pickup, and results shift to pages about collecting prescriptions on another person’s behalf. Understanding, not generation, is the entire contribution.
Related AI terms: GPT · NLP Transformer · Natural Language Processing · Sentiment Analysis · Fine-Tuning
Did you like the BERT gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About