Data Leakage
Last Updated: July 29, 2026 | By Mihail Sebastian | AI Dictionary
Information that will not exist at prediction time quietly slips into training data, so the model scores well in testing but fails in production.
What is Data Leakage?
Data leakage occurs when information that will not be available at prediction time makes its way into a model’s training data. The model scores brilliantly in testing, because it has effectively seen the answers, then fails once deployed against data that lacks them.
Leakage is dangerous precisely because everything looks right. Metrics are strong, validation passes, and the flaw only surfaces in production.
Types of Data Leakage
- Target leakage: a training feature is a proxy for, or a consequence of, the thing being predicted – like predicting loan default from a field that is only recorded after a default happens.
- Train-test contamination: information from the test set bleeds into training, for example when features are normalized or selected on the full dataset before splitting.
- Temporal leakage: a model meant to predict the future is trained on records that contain future information, a standard trap in time-series problems.
Data Leakage vs Data Drift
Data leakage is a training-time failure; data drift is a production-time failure. Leakage means the model learned from information it should never have had, so it launches already broken. Drift means the model was built correctly, but the world changed after deployment. The names sound alike; the failure modes, symptoms, and fixes have nothing in common.
| Data Leakage | Data Drift | |
|---|---|---|
| When it happens | During training | After deployment |
| Root cause | Information unavailable at prediction time enters the training data | The input distribution shifts away from the training data |
| First symptom | Test metrics look too good; production performance collapses | Production accuracy decays, often gradually |
| Remedy | Strict train/test separation, feature review | Monitoring, then retraining on recent data |
Example of Data Leakage
A bank builds a model to predict loan defaults. One feature in the training table is “number of late-payment notices sent,” and it looks powerfully predictive – the model reaches high accuracy in every offline test.
The catch: late-payment notices only exist once a loan is already going bad. At application time, when the model must actually decide, that field is always zero. The model learned to read a consequence of default as a predictor of it, and in production its accuracy collapses to little better than guessing.
Related AI terms: Data Drift · Overfitting · Model Training · Cross-Validation · Test Set
Did you like the Data Leakage gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
Mihail Sebastian — Writes about AI governance, regulation, and the technology behind them. Placeholder bio — replace with a real credential line. About