Memory Leak
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
A defect where a program keeps holding memory it no longer needs, so usage grows until the system slows or crashes – a hazard for long-running AI training jobs.
What is a Memory Leak?
A memory leak is a software defect in which a program keeps holding memory it no longer needs, so its memory use grows for as long as it runs, until performance drops or it crashes.
Short scripts finish before a leak matters. Machine learning jobs are different: model training runs for hours or days, so a tiny leak per training step compounds into a dead job – and the leak strikes scarce GPU memory as readily as ordinary RAM.
How Memory Leaks Happen
The classic cause is a lingering reference: an object gets added to a list, cache, or global that never shrinks, so even a garbage-collected language like Python cannot reclaim it. In languages with manual memory management, such as C and C++, the cause is simpler still – allocated memory that no code ever frees.
Deep learning adds its own variant. Frameworks attach a computation graph to every tensor produced during training, so storing a raw tensor for logging quietly retains the entire graph behind it, batch after batch.
The defense is the same in every case: watch memory over time, not just at startup. A flat line is healthy; a steady upward slope is a leak.
Example of a Memory Leak
A team trains a neural network in PyTorch and logs the loss each step with losses.append(loss). It looks harmless.
But loss is a tensor still attached to its computation graph, so each appended value keeps that step’s entire graph alive in GPU memory. Usage climbs a little every step. Hours into the run, the job dies with an out-of-memory error, long after anyone was watching.
The fix is one call: loss.item() extracts the plain number and lets the graph be freed. The team also adds memory tracking to their training dashboard, so the next upward slope gets caught in minutes instead of hours.
Related AI terms: GPU Memory · CUDA · Model Training · Epoch
Did you like the Memory Leak gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About