GPT (Generative Pre-trained Transformer)
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A family of language models from OpenAI that generate text one token at a time, pre-trained on vast text corpora and then adapted to specific tasks.
What is GPT (Generative Pre-trained Transformer)?
GPT (Generative Pre-trained Transformer) is a family of large language models from OpenAI that generate text by predicting the next token, one at a time, conditioned on everything that came before.
The name is the recipe: generative because it produces text, pre-trained because it first learns from an enormous body of general text, and transformer because of the architecture that does the processing.
How GPT Works
Pre-training comes first. The model reads a huge corpus of text and learns one skill: given a sequence of tokens, predict the next one. No labels are needed, so this stage is unsupervised learning at scale, and the prediction task forces the model to absorb grammar, facts, and reasoning patterns as side effects.
The transformer architecture makes this tractable. Its attention mechanism lets every token weigh its relevance to every other token in the context, so the model tracks dependencies across long passages instead of only adjacent words.
After pre-training, the raw model is adapted. Fine-tuning on instruction data and human feedback turns a next-token predictor into an assistant that follows directions rather than merely continuing text.
GPT vs BERT
The practical difference: GPT writes, BERT reads. GPT is a transformer decoder trained to predict the next word, so it generates text. BERT is a transformer encoder that reads a whole sentence in both directions at once, so it excels at understanding tasks such as search ranking and classification, but it does not generate text.
| Criterion | GPT | BERT |
|---|---|---|
| Architecture | Transformer decoder | Transformer encoder |
| Reads text | Left to right, token by token | Both directions at once |
| Training objective | Predict the next token | Predict masked-out words |
| Built for | Generating text | Understanding and classifying text |
Example of GPT
Ask a GPT-based assistant to “write a polite email declining a meeting invitation.” The model converts the request into tokens, computes a probability distribution over its entire vocabulary for the next token, picks one, appends it, and repeats. Token by token, an email assembles that exists nowhere in its training data.
ChatGPT, the chat product OpenAI built on GPT models, runs exactly this loop for a general audience. The fluency is real; the reliability is not guaranteed, because the model predicts likely text rather than verified fact, which is why hallucination is a standing risk.
Related AI terms: BERT · Generative AI · NLP Transformer · OpenAI · Fine-Tuning
Did you like the GPT (Generative Pre-trained Transformer) gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About