LSTM (Long Short-Term Memory)
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A recurrent neural network variant whose gated memory cells decide what to keep, forget, and output, letting it learn long-range patterns in sequences.
What is LSTM (Long Short-Term Memory)?
LSTM (Long Short-Term Memory) is a type of recurrent neural network whose gated memory cells choose what to remember, forget, and output at each step, so it learns long-range patterns in sequential data.
A plain RNN forgets: its gradients shrink as they flow back through many steps, the vanishing gradient problem. The LSTM’s cell state gives information a protected path through time, which is exactly what the plain RNN lacks.
How an LSTM Works
Each LSTM cell keeps a cell state, a conveyor belt of information running through the sequence, and controls it with three gates.
- Forget gate: Decides which parts of the stored state to discard, such as dropping a sentence’s subject once it is no longer relevant.
- Input gate: Decides which parts of the new input to write into the state.
- Output gate: Decides what portion of the state to expose as the step’s output.
The gates are small learned networks themselves, so the LSTM learns when to remember and forget, not just what. Training uses backpropagation through time, the same procedure as other RNNs.
Applications of LSTM
- Time series forecasting: Predicting demand, sensor readings, or prices where patterns span long horizons, still a common production use.
- Speech recognition: Mapping audio frames to text, where a phoneme’s identity depends on sounds well before it.
- Language tasks: Translation, text generation, and tagging were LSTM strongholds until transformers largely displaced them for language work after 2017.
Example of an LSTM
Ask a model to complete: “I grew up in France, moved away for work, and after twenty years abroad I still speak fluent ___.” The answer, “French”, depends on a word twenty tokens back.
A plain RNN tends to lose “France” under everything read since. An LSTM’s input gate wrote “France” into the cell state, its forget gate kept it because no later text contradicted it, and its output gate surfaces it when the prediction needs a language. That selective memory is the entire point of the architecture.
FAQ
Is LSTM still used now that transformers exist?
Yes, in narrower territory. Transformers dominate language and any task with large training budgets, but LSTMs remain in production for time series forecasting and on-device or streaming workloads, where their step-by-step processing and small footprint fit the constraints.
What is the difference between LSTM and GRU?
A GRU (Gated Recurrent Unit) is a simplified gated RNN with two gates instead of three and no separate cell state. It trains faster with fewer parameters; the two perform comparably on many tasks.
Related AI terms: Recurrent Neural Network · NLP Transformer · Sequence-to-Sequence · Vanishing Gradient
Did you like the LSTM (Long Short-Term Memory) gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About